-- Logs begin at Tue 2022-05-10 14:15:45 UTC, end at Tue 2022-05-10 15:25:13 UTC. -- May 10 14:48:12.779050 dv-cinder-839628-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 10 14:48:12.790504 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 10 14:48:12 2022] *** May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: nodename: dv-cinder-839628-2-iscsi May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: machine: x86_64 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: clock source: unix May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: pcre jit disabled May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: detected number of CPU cores: 4 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: current working directory: / May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: detected binary path: /usr/bin/uwsgi-core May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: your processes number limit is 47680 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: your memory page size is 4096 bytes May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: detected max file descriptor number: 2048 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: lock engine: pthread robust mutexes May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: thunder lock: enabled May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 10 14:48:12.797544 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 10 14:48:12.838672 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: Python main interpreter initialized at 0x55c8f85739c0 May 10 14:48:12.838672 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: python threads support enabled May 10 14:48:12.838672 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: your server socket listen backlog is limited to 100 connections May 10 14:48:12.838672 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: your mercy for graceful operations on workers is 90 seconds May 10 14:48:12.839197 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: mapped 403077 bytes (393 KB) for 2 cores May 10 14:48:12.839291 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: *** Operational MODE: preforking *** May 10 14:48:12.839651 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: *** uWSGI is running in multiple interpreter mode *** May 10 14:48:12.839651 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: spawned uWSGI master process (pid: 79925) May 10 14:48:12.839877 dv-cinder-839628-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 10 14:48:12.840433 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: spawned uWSGI worker 1 (pid: 79936, cores: 1) May 10 14:48:12.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: spawned uWSGI worker 2 (pid: 79937, cores: 1) May 10 14:48:12.840926 dv-cinder-839628-2-iscsi devstack@keystone.service[79925]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 10 14:48:18.587465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 14:48:18.587465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: from cryptography.utils import int_from_bytes May 10 14:48:18.588807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 14:48:18.588807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: from cryptography.utils import int_from_bytes May 10 14:48:18.590469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 14:48:18.590469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: from cryptography.utils import int_from_bytes May 10 14:48:18.591768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 14:48:18.591768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: from cryptography.utils import int_from_bytes May 10 14:48:18.637839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 14:48:18.638074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 14:48:18.638232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] command line args: [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 14:48:18.638502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 14:48:18.638681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ================================================================================ {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 14:48:18.638773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 14:48:18.638922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 14:48:18.639002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] admin_token = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] command line args: [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 14:48:18.639211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] config_dir = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 14:48:18.639438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ================================================================================ {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 14:48:18.639564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] config_source = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] debug = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] admin_token = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.639867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] config_dir = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] config_source = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] debug = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.642987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_config_append = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_dir = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_file = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_config_append = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_dir = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_file = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.643995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.644955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] publish_errors = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.645819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.646558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.646558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.646925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.646925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] publish_errors = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.647314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.647508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.647705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.647904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.648187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] standard_threads = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.648421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.648588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.648780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] use_journal = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] standard_threads = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.649974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] use_json = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.650172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.650422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] use_stderr = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.650593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.650786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] use_syslog = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.650980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.651228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.651401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] use_journal = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.651660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.651831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] use_json = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.651831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] use_stderr = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.652121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.652251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] use_syslog = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.652378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.652514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 14:48:18.652644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.652806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.652970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.653891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.654927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.655995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.656938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.657872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.658865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.659930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.660914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.661970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.662938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.663970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.664098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.664231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.664359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.664493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.665691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.665887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.666113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.666285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.666524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.666729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.666919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.667111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.667313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.667513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.667705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.667932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.668100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.668374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.668583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.668749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.668942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.669171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.669373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.669573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.669741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.669973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.670139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.670366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.670602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.670767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.670930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.671966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.672871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.673985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.674936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.675885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.676960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.677999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.678978 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.679897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.680967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.681889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.682021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.682150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.682277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.682996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.external = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.683944 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.password = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.684900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] auth.token = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.external = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.685857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.password = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] auth.token = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.686950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.687994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.688932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.689301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.689432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.689569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.689737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.689878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.690868 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.691927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.692871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.693875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.694913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.695970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.696910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697697 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.697956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.698095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.698784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.698926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.699993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.700928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.701955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.702984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.703914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.704969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.705889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.706917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.707988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.708920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.709971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.710909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.711961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.712999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.713923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.714873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.715936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.716997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.717911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.718873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.719989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.720916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] role.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.721962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] role.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.722856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] role.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.723966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] role.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.724937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.725962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.726953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.727917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.728886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.729883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.730893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.731855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.732952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.733941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.734945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.735922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.connection = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.736924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.737903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.738912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.connection = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.739914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.740304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.740450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.740647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.740791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.740929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.741900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79936) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.742492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 14:48:18.743629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79937) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 14:48:18.752555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.752785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.753004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.753736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.753969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.754195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.762201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.763452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.790714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.790933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.791438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.791655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.848967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.851061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.902838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.903079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.903517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.903746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.969242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.969504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.969595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.969686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.969920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.970771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.994890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.995993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.996116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:18.996181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.007382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.007610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.021890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.022896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.023156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.023239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.023239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.023362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79937) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.023426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79936) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 14:48:19.040317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.040531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 14:48:19.040803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.040905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.041089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 14:48:19.041173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 14:48:19.041337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.041418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.041582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 14:48:19.041671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.042339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.042454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.042624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.042707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.042884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.043088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.053729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.054886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.062896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.063070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.066197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.066284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.066501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.066652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.066805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.066946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.067886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.068865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.069003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.069150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.069302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.096918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.098573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.098848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.099953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.101575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.101852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.104158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.104456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.104740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.107261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.107542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.107833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.112059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.112347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.115421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.115703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.117194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.117474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.117749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.118025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.118296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.118589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.118854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.119117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.120344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.120633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.120912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.121192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.121468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.121738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.122005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.122269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.138157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.138455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.141589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.141867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.143062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.146424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.146681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.146971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.150029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.150325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.152463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.155872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.156113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.159457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.159834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.160126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.160403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.160683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.160956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.161228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.163195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.163496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.163781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.164066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.164342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.164619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.185039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.185235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.185421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.185583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.185738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.185878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.186016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.186187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.195125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.196840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.196989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.197129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.199268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.199542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.200737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.201016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.201304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.201595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.201872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.202598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.202875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.203159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.203440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.203767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.215368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.215649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.215931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.216225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.216504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.216785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.217027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.217175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.217314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.217564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.217847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.218122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.218426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.218706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.235084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.236562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.238354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.239837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.241924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.243424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.243594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.245536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.245707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.245940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.246238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.247196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.247472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.247769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.254888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.255165 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.256734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.256894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.259967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.260130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.260271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.260411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.260619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.261267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.261411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.261550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.270529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.270682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.270835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.270976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.271131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.271273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.279028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79936) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.279201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79937) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 14:48:19.279344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.279483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.279624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.279763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.280921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.281972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.282886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.283025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79936) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.283173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79937) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 14:48:19.321659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c8f85739c0 pid: 79937 (default app) May 10 14:48:19.321902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c8f85739c0 pid: 79936 (default app) May 10 14:48:19.325661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72c22-d11f-444c-a9b9-fe01fae73f11 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:19.325951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72c22-d11f-444c-a9b9-fe01fae73f11 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:19.326237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72c22-d11f-444c-a9b9-fe01fae73f11 None None] PATH_INFO: `/v3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:19.332047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1/1] 10.222.0.83 () {50 vars in 791 bytes} [Tue May 10 14:48:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:27.305463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:27.305998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:27.305998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:27.310878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1/2] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:27.316355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259cd7-133a-4904-800c-292401596a3f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:27.316832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259cd7-133a-4904-800c-292401596a3f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:27.316928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259cd7-133a-4904-800c-292401596a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:27.346644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG oslo_db.sqlalchemy.engines [None req-fe259cd7-133a-4904-800c-292401596a3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79937) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 14:48:27.527938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG passlib.handlers.bcrypt [None req-fe259cd7-133a-4904-800c-292401596a3f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79937) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 14:48:27.528354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG passlib.handlers.bcrypt [None req-fe259cd7-133a-4904-800c-292401596a3f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79937) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 14:48:27.582582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-fe259cd7-133a-4904-800c-292401596a3f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:27.584086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fe259cd7-133a-4904-800c-292401596a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:27.705811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 2/3] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 10 14:48:27.860815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:27.899644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG oslo_db.sqlalchemy.engines [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79936) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 14:48:28.048095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235ea8d-a731-494c-8b56-98710c1caecf None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:28.052466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:28.053134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:28.053419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:28.053698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:28.062037 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:28.120159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 14:48:28.120432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555c15e-cebf-46d9-99a3-45fb4f75329f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:28.132943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 2/4] 10.222.0.83 () {58 vars in 1324 bytes} [Tue May 10 14:48:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:48:28.138921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe259cd7-133a-4904-800c-292401596a3f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:28.156627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe259cd7-133a-4904-800c-292401596a3f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:28.161149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:28.162273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:28.162588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:28.162864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] PATH_INFO: `/v3/projects/06d48372060e456ea0c20a517b83a4d3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:28.164382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] RBAC: Authorizing `identity:get_project(project_id=06d48372060e456ea0c20a517b83a4d3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:28.227164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 14:48:28.227514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f2be8-561f-4c33-9ae4-2463d8d16da1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:28.233717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 3/5] 10.222.0.83 () {56 vars in 1207 bytes} [Tue May 10 14:48:28 2022] GET /identity/v3/projects/06d48372060e456ea0c20a517b83a4d3 => generated 344 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:30.497142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292376e-a8a0-4a1b-83e5-e3ed081fcc62 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:30.497692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292376e-a8a0-4a1b-83e5-e3ed081fcc62 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:30.497692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292376e-a8a0-4a1b-83e5-e3ed081fcc62 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:30.501846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 3/6] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:30.507378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:30.507808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:30.507940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:30.528617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:30.530121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:30.537797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 4/7] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 10 14:48:30.559519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292376e-a8a0-4a1b-83e5-e3ed081fcc62 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:30.571142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292376e-a8a0-4a1b-83e5-e3ed081fcc62 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:30.575748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:30.576364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:30.576644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:30.576915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:30.584265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:30.586022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f4371-bb70-4c69-82b9-d74a5f3379ec admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:30.595064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 4/8] 10.222.0.83 () {58 vars in 1324 bytes} [Tue May 10 14:48:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:48:30.600130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:30.611147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4359306-56d9-49bb-89b2-8194d9d3208a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:30.616311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:30.617195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:30.617479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:30.617756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:30.619177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:30.621057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d424a-2e0a-499e-ae48-d2ba55317289 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:30.622719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 5/9] 10.222.0.83 () {56 vars in 1198 bytes} [Tue May 10 14:48:30 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:32.887032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1f1f-2b3c-45e0-aba1-f32f1d0c2371 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:32.887551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1f1f-2b3c-45e0-aba1-f32f1d0c2371 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:32.887667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1f1f-2b3c-45e0-aba1-f32f1d0c2371 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:32.888939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 5/10] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:32.894047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a551-1488-463f-82da-aee319723274 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:32.894450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a551-1488-463f-82da-aee319723274 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:32.894634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a551-1488-463f-82da-aee319723274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:32.915508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7495aa-5092-45c9-8d14-0690550b1bc6 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:32.915844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7495aa-5092-45c9-8d14-0690550b1bc6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:32.916074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7495aa-5092-45c9-8d14-0690550b1bc6 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:32.917468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 6/11] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:48:32.923970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:32.924247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:32.924527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:32.965736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG passlib.handlers.bcrypt [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79936) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 14:48:32.966159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG passlib.handlers.bcrypt [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79936) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 14:48:33.013746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:33.015388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:33.024549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-dcc7a551-1488-463f-82da-aee319723274 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:33.025708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-dcc7a551-1488-463f-82da-aee319723274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:33.102319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 6/12] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:33.111047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 7/13] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:33.125599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7a551-1488-463f-82da-aee319723274 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.132170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.220090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7a551-1488-463f-82da-aee319723274 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.224141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f6c80-6241-4da5-8859-06fff97d252c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.252399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.253184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.253466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.253738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.255938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.256817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.257105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.257381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.264371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:33.275540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:33.277192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:48:33.278013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 7/14] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:33.282900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.320326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.321827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.376477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efe2b5-e15a-4a3e-bbeb-766531c8ce9a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.408126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.408772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.409055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.409328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.410064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:33.410369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.411773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.412314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 8/15] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:48:33.418247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.423988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 8/16] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:33.429553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.509408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a2952-1387-40b1-85a5-f5807b30e4db None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.523005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ac63e-b8b6-4d2e-a58d-cfd99a9a1176 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.540927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.541748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.542031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.542305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.555076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.555981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.556263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.556538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.557814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.559354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.561052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 9/17] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:33.566824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.667352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.667962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f9c6-1f8c-4aab-ab19-eb2aaefe9a86 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.668809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:48:33.689366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:33.690330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:33.691210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:48:33.691210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:48:33.691210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:48:33.691210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:33.691210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 9/18] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:48:33.696399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.700349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.701145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.701427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.701703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.702903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.704362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.705943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 10/19] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:33.711400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:33.790476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81804bd-09da-40f2-87e2-35eedaeb39bc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.803502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaca195-5e8e-46fd-ba93-97ae9766c1a2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:33.822249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.823486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.823772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.824042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.824775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:33.825082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.826458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d44b3e-432d-4278-8571-65cc7c3bf8c1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.835429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:33.836071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:33.836353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:33.836636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:33.837453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.domain.id=default, user.id=ce068d36274d4c2ab89128932ddb806d) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:33.837761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:33.839241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6fafe-2571-4569-b700-c2ed6e10ec9a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:33.843320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 10/20] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:33.845834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 11/21] 10.222.0.83 () {56 vars in 1322 bytes} [Tue May 10 14:48:33 2022] GET /identity/v3/role_assignments?user.id=ce068d36274d4c2ab89128932ddb806d&scope.domain.id=default&role.id=57e23e25b8c046ba882d07965d852386 => generated 233 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:36.478680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfceae-96de-41e3-b6b3-e9d89f949cb8 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:36.479245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfceae-96de-41e3-b6b3-e9d89f949cb8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:36.479245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfceae-96de-41e3-b6b3-e9d89f949cb8 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:36.480925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 11/22] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:36.486347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:36.486763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:36.486956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:36.504327 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:36.505515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:36.582587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 12/23] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:36.600088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfceae-96de-41e3-b6b3-e9d89f949cb8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:36.723944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfceae-96de-41e3-b6b3-e9d89f949cb8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:36.769369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:36.770091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:36.770374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:36.770670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:36.787347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:36.788856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ebe6a55d-99c9-4d84-9e1d-06744dc55d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:48:36.809347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:36.810209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:36.811069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:48:36.811069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:48:36.811069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:48:36.811069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:36.814532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 12/24] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:36.816724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:36.836711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc4a3-565a-411b-a773-5f53f740f452 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:36.836990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc4a3-565a-411b-a773-5f53f740f452 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:36.837271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc4a3-565a-411b-a773-5f53f740f452 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:36.839188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 13/25] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:48:36.844210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199bcec-9529-48b1-94a1-f0a1a2787773 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:36.844625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199bcec-9529-48b1-94a1-f0a1a2787773 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:36.844755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199bcec-9529-48b1-94a1-f0a1a2787773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:36.870377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5199bcec-9529-48b1-94a1-f0a1a2787773 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:36.871578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5199bcec-9529-48b1-94a1-f0a1a2787773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:36.957875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0174eb-63b5-4daa-99df-1960fce75913 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.003483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.004156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.004437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.004712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.005437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:37.005743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:37.007154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:37.028212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 14/26] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:37.029526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109efb-b67f-4d16-8c42-d29a4983367a None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.029819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109efb-b67f-4d16-8c42-d29a4983367a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.030095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109efb-b67f-4d16-8c42-d29a4983367a None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.031377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 15/27] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:37.040130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.040417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.040699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.046143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 13/28] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:37.051850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.077351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:37.078561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:37.183407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff126e65-a432-48fb-ab4a-d082455fd637 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.200718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 16/29] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:37.201553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.220466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.221153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.221435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.221712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.222465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:37.223986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:37.366962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 14/30] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/projects => generated 301 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:37.368141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.394215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3b7f-40af-4e35-a89b-403c12b69be3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.476325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac2f4f-4ff6-4508-9aa8-7eeecf269b66 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.482257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.482991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.483279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.483558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.484328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:37.485791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b65bc-8de0-402c-a7f9-8b8ff1d21acc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:37.525259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.526685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.526975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.529154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.549739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-9c83183d-7c12-4ed0-a56b-3655df412a3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:37.558410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:37.559371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:37.560166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:48:37.572424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 15/31] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:37.574527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4fcef-f484-493e-8a19-18928be80b6d None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.574819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4fcef-f484-493e-8a19-18928be80b6d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.575102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4fcef-f484-493e-8a19-18928be80b6d None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.576871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 16/32] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:37.578650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4fcef-f484-493e-8a19-18928be80b6d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.677964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 17/33] 10.222.0.83 () {60 vars in 1131 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/roles => generated 219 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:37.679561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6e89-8638-4cd9-ae87-5e53dfcf7211 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.679855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6e89-8638-4cd9-ae87-5e53dfcf7211 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.680140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6e89-8638-4cd9-ae87-5e53dfcf7211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.715990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-487a6e89-8638-4cd9-ae87-5e53dfcf7211 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:37.717534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-487a6e89-8638-4cd9-ae87-5e53dfcf7211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:37.740732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4fcef-f484-493e-8a19-18928be80b6d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.798171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.798900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.799188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.799467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.800238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:37.800547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:37.802312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349be6-9ec2-43cc-b8c7-96d52bd0f994 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:37.811269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 17/34] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:37.812629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b91581-88ca-48a8-8a78-d51ce9b87e8a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.812984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b91581-88ca-48a8-8a78-d51ce9b87e8a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.813186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b91581-88ca-48a8-8a78-d51ce9b87e8a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.818904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 18/35] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:37.825864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b91581-88ca-48a8-8a78-d51ce9b87e8a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.829282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 18/36] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:37.831036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.831558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.832081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.860719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:37.862055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:37.945609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b91581-88ca-48a8-8a78-d51ce9b87e8a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:37.961995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 19/37] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:37.963003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:37.993751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:37.996284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:37.996817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:37.997328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:37.999880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.002216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.004496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 19/38] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:37 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:38.007414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:38.073443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593caa6b-d083-4915-b1fb-8723e9d41b9e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.106007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.107044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.107529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.107999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.109516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.109747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c8a6-d512-4bf9-9ead-00acc8f384d7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.111468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.125896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 20/39] 10.222.0.83 () {60 vars in 1131 bytes} [Tue May 10 14:48:37 2022] POST /identity/v3/roles => generated 225 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:38.130459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:38.156184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.157070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.157974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.158256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.160072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.162212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.164321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 20/40] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:38.170847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:38.246594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6003-eedd-4287-8459-3685d1b7383f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.275385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb61e9-68b8-4f9b-b922-7fdab2b82549 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.280448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.281302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.281593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.281871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.283446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.284943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62066-1444-4771-9aa2-b77ae38bac52 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.291993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 21/41] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:38.291993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa215-00a6-4b90-9a81-3e921bae9325 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.291993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa215-00a6-4b90-9a81-3e921bae9325 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.291993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa215-00a6-4b90-9a81-3e921bae9325 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.291993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 22/42] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:38.295605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa215-00a6-4b90-9a81-3e921bae9325 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:38.317789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.318587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.319674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.319674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.319869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.321294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278c3d-b7df-4eb3-9352-262605c2ac23 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.347700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 21/43] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:48:38 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:38.348994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.349288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.349573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.367749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:38.369362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:38.409139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa215-00a6-4b90-9a81-3e921bae9325 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.444331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.451139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 22/44] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:38.451366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.451366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.451366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] PATH_INFO: `/v3/domains/default/users/ce068d36274d4c2ab89128932ddb806d/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.453787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ce068d36274d4c2ab89128932ddb806d, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.455341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c4558-9976-457f-81e2-078a64578b5e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.471402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:38.472554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 23/45] 10.222.0.83 () {58 vars in 1361 bytes} [Tue May 10 14:48:38 2022] PUT /identity/v3/domains/default/users/ce068d36274d4c2ab89128932ddb806d/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:38.581041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b28e13-8c2e-4c5d-b5e8-515c113d729b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:38.617494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:38.618178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:38.618504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:38.618785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:38.619582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:38.620993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38ced1-2c2a-4c5c-b766-108dc211cf7f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:38.636199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 23/46] 10.222.0.83 () {60 vars in 1131 bytes} [Tue May 10 14:48:38 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:48:40.621346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef234e-d5a4-462e-b176-3a1b4e2332cf None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.621866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef234e-d5a4-462e-b176-3a1b4e2332cf None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.621974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef234e-d5a4-462e-b176-3a1b4e2332cf None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.623308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 24/47] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:40.628689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.629090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.629258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.646591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:40.648646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:40.704623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930e4db-358f-4130-b204-73c9fb47de29 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.704911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930e4db-358f-4130-b204-73c9fb47de29 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.705195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930e4db-358f-4130-b204-73c9fb47de29 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.706594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 25/48] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:48:40.711352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.711708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.711860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 24/49] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:40.712266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.728699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:40.730501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:40.736744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:40.800903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 26/50] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:40.823554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:40.828951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a75bb-2041-4c1d-9bd8-1c589c6675e4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:40.862152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:40.862985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.863271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.863551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.864734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:40.866455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:40.868151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 25/51] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:40.873574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:40.917996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2209e0-89c2-44da-8dca-ac6a7d09f531 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:40.950845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:40.951640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.951920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.952194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:40.962592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:40.966379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea7d3c-4988-431e-a6bd-da850d66ca10 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:40.971914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:40.972922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:40.973788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 27/52] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:48:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:48:40.978634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:40.998314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:40.999002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:40.999286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:40.999564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:41.000271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:41.001743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:41.026836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 26/53] 10.222.0.83 () {60 vars in 1140 bytes} [Tue May 10 14:48:40 2022] POST /identity/v3/projects => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:48:41.074526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e8859-4c00-4f96-8ee9-5916eb83897e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:41.109261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:41.109950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:41.110236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:41.110549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:41.111321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:41.111631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:41.113039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:41.127496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 28/54] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:41.133109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:41.230813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6482f-6a57-41b5-86b7-c23aa666e76d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:41.262936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:41.263800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:41.264077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:41.264348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:41.265526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:41.267024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:41.268854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 27/55] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:41 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:41.274068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:41.366767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8281-43cb-4404-a18a-4cc5c9c3bc2c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:41.398016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:41.398769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:41.399053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:41.399328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:41.400477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:41.402414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105445-f212-45fe-85d1-a88e11b7e295 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:41.404028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 29/56] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:41.409640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:41.504810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f13b-9ff4-49ea-b6de-a825c33497d7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:41.536866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:41.537510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:41.537791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:41.538065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:41.538922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.domain.id=default, user.id=ce068d36274d4c2ab89128932ddb806d) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:41.539229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:41.540666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc942b-edbc-4198-8596-0a30436af001 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:41.546819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 28/57] 10.222.0.83 () {56 vars in 1322 bytes} [Tue May 10 14:48:41 2022] GET /identity/v3/role_assignments?user.id=ce068d36274d4c2ab89128932ddb806d&scope.domain.id=default&role.id=57e23e25b8c046ba882d07965d852386 => generated 530 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:43.270220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00914def-f4a0-42fd-9f10-813ff39a101f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:43.270740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00914def-f4a0-42fd-9f10-813ff39a101f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:43.270740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00914def-f4a0-42fd-9f10-813ff39a101f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:43.272086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 30/58] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:43.277714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:43.278119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:43.278271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:43.294669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:43.295867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:43.357719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 29/59] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:43.381675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00914def-f4a0-42fd-9f10-813ff39a101f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:43.473266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00914def-f4a0-42fd-9f10-813ff39a101f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:43.504765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:43.505532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:43.505808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:43.506077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:43.507245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:43.508675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0c86a-9b7f-4dd5-a312-695ed7672374 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:43.510272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 31/60] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:43.515453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:43.607515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e3594-5dee-4eee-aa89-149f5e352c67 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:43.639082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:43.639748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:43.640036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:43.640315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:43.641027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:43.642510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e00fbf-14a4-4618-bdf7-0b09d2d8359a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:43.669567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 30/61] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:48:43 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:48:47.106422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8fd2-aafa-4811-9255-b186e0641427 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.106422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8fd2-aafa-4811-9255-b186e0641427 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.106422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8fd2-aafa-4811-9255-b186e0641427 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.107712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 32/62] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:47.113244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.113985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.114435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.139299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:47.140755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:47.160788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b513-8427-4e5e-9ac1-f93172a4182f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.161410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b513-8427-4e5e-9ac1-f93172a4182f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.161902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b513-8427-4e5e-9ac1-f93172a4182f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.164003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 33/63] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:47.170271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.170847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.171324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.190853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:47.192389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:47.247249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 31/64] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:47.277018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 34/65] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:47.292026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.361122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.415829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfbbc-3b29-4023-8fc6-2d22b3929224 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.451379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.452086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.452534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.452971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.464253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:47.470096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02790-8472-4caa-997d-5f18adb2ccdf None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e9afeac3-a166-4ab0-afbf-a4a568eda6ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:47.472426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:47.473307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:47.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:48:47.476509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 32/66] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:47.478020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8713f03-a750-4732-8a23-85d535e7ce54 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.478537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8713f03-a750-4732-8a23-85d535e7ce54 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.479587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8713f03-a750-4732-8a23-85d535e7ce54 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.481071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 33/67] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:47.485942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8713f03-a750-4732-8a23-85d535e7ce54 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.505407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.506215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.506524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.506966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.517506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-5698f44d-f51f-4932-b1f9-a383e01886c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:47.525469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:47.526931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:47.527703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:47.530207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 35/68] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:48:47.531640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.532091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.532531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.555500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:47.557399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:47.594808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8713f03-a750-4732-8a23-85d535e7ce54 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.621801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 36/69] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:47.623347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.627402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.628056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.628606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.629333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:47.629636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:47.631046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b68cfa-c8f3-42a6-aa89-c37fbbfb9a35 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:47.643002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 34/70] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:47.643002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f34a5-0bbe-4a20-8d72-1d2aa1172dd2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.643002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f34a5-0bbe-4a20-8d72-1d2aa1172dd2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.643002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f34a5-0bbe-4a20-8d72-1d2aa1172dd2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.643002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 35/71] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:47.643903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32951a-1985-4cd4-8b5f-0d373cb5f6be None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.643903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32951a-1985-4cd4-8b5f-0d373cb5f6be None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.644087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32951a-1985-4cd4-8b5f-0d373cb5f6be None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.646423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 36/72] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:47.646423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32951a-1985-4cd4-8b5f-0d373cb5f6be None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.720593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1f3ce-6b62-4efa-a476-14e08bdda66d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.740688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32951a-1985-4cd4-8b5f-0d373cb5f6be None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.752261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.752932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.753219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.753497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.754241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:47.754788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:47.755954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:47.764258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 37/73] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:47.769432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.772864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.773739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.774021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.774297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.787408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:47.788865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c91d2-6efc-4b74-9001-6dc2f85410ac None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:47.790526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 37/74] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:47.792800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256901f4-8540-4a97-acad-5877ba718251 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.793089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256901f4-8540-4a97-acad-5877ba718251 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.793367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256901f4-8540-4a97-acad-5877ba718251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.808888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-256901f4-8540-4a97-acad-5877ba718251 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:47.810059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-256901f4-8540-4a97-acad-5877ba718251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:47.861114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501e4d-a924-4099-8b2c-398b6bc586cb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:47.871005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 38/75] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:47.872859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.873157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.873443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.890109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:47.891350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:47.895635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:47.896444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:47.896732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:47.897013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:47.909725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:47.917824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:47.918831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:47.919593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 38/76] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:47.920241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:47.959062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 39/77] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:47.959062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.014486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68389e1c-e945-42ee-ac7b-90e36f8d90ff None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.045530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.046416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.046718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.046996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.048191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.049624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.051264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 39/78] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.055754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a3b3c-a357-4cba-97f8-9e58f49f2e0b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.055837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.085198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.085846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.086131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.086422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.143197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e661be-ed69-4d96-90ed-3effb258866f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.149190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.150708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.173701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.174501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.174782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.175055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.185044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:48.192947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:48.193933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:48.194756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 40/79] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:48.195378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.247905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 40/80] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:48.247905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.286212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807be08-d2f4-4b7d-bb49-7dfcd386702e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.317913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.318610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.318892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.319168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.320763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:48.321069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.323573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.331368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 41/81] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.332225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.345170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb53d0-346f-4d9c-9380-d03e87789845 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.376708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.377499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.377781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.378054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.379258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.380696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.382282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 41/82] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.383160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.425622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f700d-94fc-4913-b160-d90c04aed0cb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.456936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.457573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.457852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.458126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.474870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb7d0b-f404-4e37-a180-e319b66a0ffa None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.508138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.508790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.509073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.509345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.510101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:48.510424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.511788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.519335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 42/83] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.520185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.524836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.526349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.611586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d6b5-b0d8-4dd4-bdbe-3584b95a66e0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.615996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 42/84] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:48.616798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.643863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.644720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.645001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.645275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.646471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.647908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.649577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 43/85] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.650300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.707448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c823b3-5eb8-4d71-b68e-3a300bd6fd19 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.738624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.739545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.739824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.740095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.741534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.742338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f62bff-9bc8-4f24-8333-1f5c3299e5da None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.743001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.745250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 43/86] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.747119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.774594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.775273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.775563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.775838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.776544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.777988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.797613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 44/87] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:48:48 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:48.798551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.856631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674263ab-b8d7-41f1-b82f-dbee28376718 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.891481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.892290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.892577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.892856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.894128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.895940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.898048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 44/88] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.898967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:48.909133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87dd2-f8e7-42eb-a39f-4a89d01b1358 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:48.944240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:48.945306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:48.945695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:48.946075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:48.947548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:48.949141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:48.956393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 45/89] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:48.956393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.003528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d8661-6260-45c7-887d-e220b1f40ff3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.038762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.039429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.039715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.039994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.041269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:49.041269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.043200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.051046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 45/90] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:49.051766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.061532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772748b-b213-41b2-a8a2-a166ce65f731 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.099503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.100141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.100425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.100701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.162110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c02b-4cfd-4226-a916-6309f9b91841 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.176885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.178589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.200659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.201693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.202190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.202694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.273485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.275326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.284015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 46/91] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:49.284718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.375492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 46/92] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:49.376674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.388267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad292f0-10cf-4a2c-972d-d35fd2560c13 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.423402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.423823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.424155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.424443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.425285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=ce068d36274d4c2ab89128932ddb806d) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:49.425829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.427151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.432713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 47/93] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/role_assignments?user.id=ce068d36274d4c2ab89128932ddb806d&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:49.433508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.508408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2d30-42eb-4dd1-8a00-0a465589b301 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.539620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0a2-a577-4c57-9ef9-3acb7c6731a0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.564223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.565520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.566001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.566536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.568180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.569958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.573062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 47/94] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:49.579224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.581099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.581987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.582270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.582579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] PATH_INFO: `/v3/projects/b469217e345f4099b257694853fed685` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.583949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] RBAC: Authorizing `identity:get_project(project_id=b469217e345f4099b257694853fed685)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.585408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.587154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 48/95] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/projects/b469217e345f4099b257694853fed685 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:49.595053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:49.743210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2bb0-2527-4876-b180-a5d30de4ad0b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.806244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.807509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.808262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.808549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.809690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:49.810224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.815707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd721b30-5d29-452b-af14-e427d48711e4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.821727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50746d72-3007-4633-b4ae-29a05e790f99 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:49.825547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 48/96] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:49.866377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:49.867132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:49.867415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:49.867693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:49.868547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=b469217e345f4099b257694853fed685, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:49.868859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:49.870955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78405020-a1fd-486a-9e37-d187e9101466 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:49.877451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 49/97] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:49 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=b469217e345f4099b257694853fed685&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:51.698519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3c40-066a-4a8a-96a3-4ba8d18f80d5 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:51.699044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3c40-066a-4a8a-96a3-4ba8d18f80d5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:51.699119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3c40-066a-4a8a-96a3-4ba8d18f80d5 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:51.700569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 49/98] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:51.709709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:51.710259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:51.710259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:51.733263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:51.734492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:51.823277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 50/99] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:51.859565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa3c40-066a-4a8a-96a3-4ba8d18f80d5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:51.985192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa3c40-066a-4a8a-96a3-4ba8d18f80d5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.033655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.034475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.034760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.035036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.048202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-2e3b4a87-01a0-449d-b2f0-a8a35e8dc50a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:52.056129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:52.057548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:52.058317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 50/100] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:52.063891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.122140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd125b1-11d8-4eab-81fe-916a550e88ac None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.122469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd125b1-11d8-4eab-81fe-916a550e88ac None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.122716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd125b1-11d8-4eab-81fe-916a550e88ac None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.124218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 51/101] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:48:52.129429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f18cd-cf3c-4dfd-a54b-a49cd1d66aa7 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.129762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f18cd-cf3c-4dfd-a54b-a49cd1d66aa7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.130040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f18cd-cf3c-4dfd-a54b-a49cd1d66aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.151416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-255f18cd-cf3c-4dfd-a54b-a49cd1d66aa7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:52.152944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-255f18cd-cf3c-4dfd-a54b-a49cd1d66aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:52.209312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91b6ea-9cd5-4bcf-ac21-cb17074f8cee None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.253434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.254041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.254323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.254625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.255380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:52.255688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:52.257065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c28c0f-9802-4bce-b3ab-b3e5b0a5b3fe None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:52.275738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 51/102] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:52.276835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa60ec-ac31-4155-ab21-37bedcdbafc1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.277119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa60ec-ac31-4155-ab21-37bedcdbafc1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.277393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa60ec-ac31-4155-ab21-37bedcdbafc1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.278778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 52/103] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:52.281614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa60ec-ac31-4155-ab21-37bedcdbafc1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.296404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 52/104] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:52.297695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.297982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.298259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.330112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:52.331615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:52.408470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 53/105] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:52.408470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.506674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa60ec-ac31-4155-ab21-37bedcdbafc1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.517015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89e58b-4312-4e6f-801b-a54e2fac3ce5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.559203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.562184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.562493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.562771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.565632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:52.570218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:52.572049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 54/106] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:52.572683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.582196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.583119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.583400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.583750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] PATH_INFO: `/v3/users/59a143c30197428cbbf14e9986b00bf7` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.623976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] RBAC: Authorizing `identity:get_user(user_id=59a143c30197428cbbf14e9986b00bf7)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:52.625464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:52.631310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 53/107] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/users/59a143c30197428cbbf14e9986b00bf7 => generated 289 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:52.632188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.699914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b96800-0819-412a-bc7b-165bbc76f65d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.748510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53cbd2-9d31-4732-b836-fa500597b1b2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.775270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.776587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.776865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.777140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.788225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:52.792168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:52.792729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:52.793479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:52.795216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:52.797112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:52.806623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:52.815500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:52.816411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:52.817218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:52.820303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 55/108] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:52.825905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.849835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 54/109] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:48:52 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:52.850644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:52.975409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e005-8687-4b0b-8dd6-14eb8bb85171 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:52.998583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716a15c-ef25-4ef7-9c89-65d7f08fc888 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.021492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.022543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.022967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.023434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.042659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.043599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.044027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.044451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.045349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:53.045808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.047404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85ad2-1927-4e6c-a2b1-ad06189d755d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.062525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 55/110] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:53.064128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2debc-9326-4f8d-bee9-61cd97168dc1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.064584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2debc-9326-4f8d-bee9-61cd97168dc1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.065015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2debc-9326-4f8d-bee9-61cd97168dc1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.071140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 56/111] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:53.073178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2debc-9326-4f8d-bee9-61cd97168dc1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.105211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.106724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbabb9-c271-47ba-90af-e81075022bfd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.210033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 56/112] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:53.211646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.211934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.212213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.232871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:53.234419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:53.250932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2debc-9326-4f8d-bee9-61cd97168dc1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.294680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.295553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.296348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.296640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.298166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.300515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7da8e-bccc-4741-a955-ef281b83e62e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.303084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 57/113] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:53.304531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b49b3-d284-42bf-9583-a5573a28eca1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.305136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b49b3-d284-42bf-9583-a5573a28eca1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.305696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b49b3-d284-42bf-9583-a5573a28eca1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.307546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 58/114] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:53.308495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b49b3-d284-42bf-9583-a5573a28eca1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.310461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 57/115] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:53.311422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.430204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b49b3-d284-42bf-9583-a5573a28eca1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.439469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4af7e3-3a26-4aa7-b207-379c4f0851b4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.465924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.467402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.467691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.467972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.469620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.471454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4ca9-2aaa-4157-88b0-3ef0a81e2491 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.473358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 59/116] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:53.474170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.475092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.475380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.475661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.476963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.478655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.479750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.480311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.481280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 58/117] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:53.482047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.501224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:53.502429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:53.573691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 60/118] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:53.579187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.612733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03582b2d-7e54-4ff6-a4e1-0aca12652ed3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.648148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.648963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.649250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.649535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.660688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:53.669749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:53.670646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:53.671473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 59/119] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:53.672756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.714954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5564c47-fff7-455f-b032-47ceb87c9fc1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.752576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.753275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.753516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.753795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.754653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=59a143c30197428cbbf14e9986b00bf7) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:53.754969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.756437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.762571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 61/120] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/role_assignments?user.id=59a143c30197428cbbf14e9986b00bf7&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:53.763360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.784718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea87b-693f-4672-9f31-4d8bf6521e06 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.837586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.839109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.840535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.841067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/users/9b05b527bfb04ddb84873f9672ebcae0/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.843579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, user_id=9b05b527bfb04ddb84873f9672ebcae0, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:53.846094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:53.864956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 60/121] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:48:53 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/users/9b05b527bfb04ddb84873f9672ebcae0/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:53.866893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:53.878993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0d863-cd82-4e64-bfc1-59ffd0004187 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:53.922371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:53.923518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.923518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.923789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.937937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-05dba44a-45eb-438f-8224-76d9d677af14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:53.949764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:53.950787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:48:53.951556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 62/122] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:53.952987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377b627-93fd-4a0b-ab61-038cc7404f87 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.952987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377b627-93fd-4a0b-ab61-038cc7404f87 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.953161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377b627-93fd-4a0b-ab61-038cc7404f87 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:53.958578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 63/123] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:53.961362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:53.961793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:53.962196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.001826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:54.004510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:54.040705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5574e4e-09ca-4aaf-9227-c8cd6c43083d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.120057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.120747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.121032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.121425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.121837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 64/124] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:54.122054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:54.122361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.123430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.123778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.132174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 61/125] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:48:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.137724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.241516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4af81-31c7-4b59-9ffd-379cced39933 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.247809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b618f-f710-4f15-b125-688deba7f139 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.285800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.286694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.287141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.287554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.287729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.288016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.288303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.288468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.289534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:54.289938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.290098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.292136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.292443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.294256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 62/126] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:54.295214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.301276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 65/127] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.302101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.400153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7fd79-14d7-439a-aba6-70a81e7650bc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.434240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.435278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.435655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.436025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.437321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.438915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.439658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434006d-d83a-4df5-8f69-3a7c71ed19bd None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.440972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 66/128] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.441667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.500866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.501949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.502381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.502822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.519338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:54.528024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:54.528904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:54.529688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:54.532049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 63/129] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:54.533019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.573023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950cd0-615a-431f-b438-9ef7c21a341b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.633111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e831f0-6e56-45e7-a7fd-adbf72d8e816 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.665726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.666709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.666975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.667255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.668554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.669030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.670003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.670382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.670894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.671365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/users/9b05b527bfb04ddb84873f9672ebcae0/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.671634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 67/130] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.672498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.674259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, user_id=9b05b527bfb04ddb84873f9672ebcae0, role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.675947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.695911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 64/131] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:48:54 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/users/9b05b527bfb04ddb84873f9672ebcae0/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:54.697132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.878226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706799d5-1d2e-4630-a182-dac62f335edc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.886995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538b15-43c7-4e86-8ca4-04b7e7a9c467 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:54.971241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.971875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:54.972572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.972860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.973143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.973419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:54.973898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:54.973996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:54.974214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.974679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:54.975679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.976503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:54.978189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:54.980822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 65/132] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.987314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:54.991104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 68/133] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:54.997061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:55.119051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d532-496d-482a-93f0-0ca4b03b21de None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:55.139496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916619b-3f41-4e72-81f4-fffc6947fad3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:55.171119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:55.172374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.172657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:55.173137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.173583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.173811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.174054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/users/ce068d36274d4c2ab89128932ddb806d/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.174159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.175400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:55.176698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, user_id=ce068d36274d4c2ab89128932ddb806d, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:55.176962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:55.178598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:55.178719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 69/134] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:54 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:55.187787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:55.207328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 66/135] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:48:54 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/users/ce068d36274d4c2ab89128932ddb806d/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:48:55.322168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa5f19-17fd-4fb7-9c70-8c86bd0178fd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:55.357175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:55.358096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.358379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.358677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] PATH_INFO: `/v3/projects/b469217e345f4099b257694853fed685` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.359870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] RBAC: Authorizing `identity:get_project(project_id=b469217e345f4099b257694853fed685)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:55.361336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d86c63-6262-481b-8efa-bc47b5207a12 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:55.363061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 70/136] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:55 2022] GET /identity/v3/projects/b469217e345f4099b257694853fed685 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:55.368986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:55.610358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679cb73-d1db-4630-b0f5-9c6b90d06f7d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:55.655928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:55.657230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.657739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.658237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] PATH_INFO: `/v3/projects/b469217e345f4099b257694853fed685/users/9b05b527bfb04ddb84873f9672ebcae0/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.660523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b469217e345f4099b257694853fed685, user_id=9b05b527bfb04ddb84873f9672ebcae0, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:55.662194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c815286-40ed-49eb-bec7-3a5fb372bfd7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:55.690653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 67/137] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:48:55 2022] PUT /identity/v3/projects/b469217e345f4099b257694853fed685/users/9b05b527bfb04ddb84873f9672ebcae0/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:55.802709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5004-d8c9-4ae5-88b1-a851e411f5dd None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.803091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5004-d8c9-4ae5-88b1-a851e411f5dd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.803253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5004-d8c9-4ae5-88b1-a851e411f5dd None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.804903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 71/138] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:55.818801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:55.819536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:55.820031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:55.855875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:55.857475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:55.967743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 68/139] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:55.999945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5004-d8c9-4ae5-88b1-a851e411f5dd None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:56.127858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5004-d8c9-4ae5-88b1-a851e411f5dd None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:56.167256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:56.168111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:56.168342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:56.168616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:56.169786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:56.171356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc11d9-69e9-44fb-9bd3-0b6cfd657a62 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:56.173276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 72/140] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:56.178336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:56.312274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665315b-41b2-4d3a-872e-f5193abbb20d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:56.350097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:56.350798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:56.351081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:56.351355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:56.352083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:56.353550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b560495-08f2-4fb5-8c98-168c430e12f5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:56.390177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 69/141] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:48:56 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:57.323783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa60b-acf0-4042-992c-b1af4ccf6e4e None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.324279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa60b-acf0-4042-992c-b1af4ccf6e4e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.324279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa60b-acf0-4042-992c-b1af4ccf6e4e None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.325608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 73/142] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:57.331075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.331578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.331740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.354912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:57.356090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:57.359841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a366aa-ffe9-4f5e-a275-e7ef39368906 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.360128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a366aa-ffe9-4f5e-a275-e7ef39368906 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.360405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a366aa-ffe9-4f5e-a275-e7ef39368906 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.361947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 74/143] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:57.375274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.376042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.376516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.398073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:57.400946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:57.459909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 70/144] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:57.483507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 75/145] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:57.507006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:57.531277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:57.640628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a8267-b9c9-42e0-88aa-6c15745fc1f7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:57.649575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2d344-8f39-417f-8161-085aa2bd9b19 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:57.687377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:57.688223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.688503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.688779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.709377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:57.710251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.710557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.711218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.712226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:57.725307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:57.727854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:48:57.729756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 76/146] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:48:57.736237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:57.737718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:57.759150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:57.761789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:57.764147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 71/147] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:57.768596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:57.919112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d990e-34c2-4bff-8cc7-d55cd6018588 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:57.957770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:57.957770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.957770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.957770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.958561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:57.958561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:57.959903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fe8d1-d249-463d-b617-b6e14116bf47 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:57.973650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 77/148] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:48:57.975365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867bbf-88eb-46c3-9ac5-aa753cbfd3f5 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.975654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec05434-5138-4ed6-9022-486af06666c0 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:57.976108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867bbf-88eb-46c3-9ac5-aa753cbfd3f5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.976591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867bbf-88eb-46c3-9ac5-aa753cbfd3f5 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:57.978022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 78/149] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:57.983521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:57.984083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:57.984555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.012122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:58.013632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:58.026564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.027607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.028089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.028558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.029511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] RBAC: Adding query filter params (name=member) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:58.030023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:58.031641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:58.048063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 72/150] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:58.050743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.152758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 79/151] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:58.154075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.182793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458af5c6-7e42-4c08-9c81-1f6d641ece69 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:58.221462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.222621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.223053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.223474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] PATH_INFO: `/v3/users/59a143c30197428cbbf14e9986b00bf7` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.230183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] RBAC: Authorizing `identity:get_user(user_id=59a143c30197428cbbf14e9986b00bf7)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:58.231865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:58.234017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 73/152] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/users/59a143c30197428cbbf14e9986b00bf7 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:58.234959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.330228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181d50d-26b0-4a29-beab-375f664fa390 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:58.346735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72d875-5b1c-40ae-96fe-26b2a4e28e8b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:58.377906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.379040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.379458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.379966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.381656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:58.383727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:58.385990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 80/153] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:58.387662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.389180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.390226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.390741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.391155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.414335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:58.425971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:58.427130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:48:58.428070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:58.430329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 74/154] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:58.432294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.617427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afec328-7ae1-4ece-aa5d-29b5d0b9e19b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:58.628591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020440-9097-4115-8ff0-0ff338e67d68 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:58.689934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.690642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.690919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.691200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.704477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:58.705347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:58.705627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:58.705899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:58.709399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:58.710883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:58.712762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 81/155] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:58.713909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.787706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:58.789173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:58.922298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 75/156] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:48:58.925780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:58.976507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0c9a7-8221-4f52-8163-9b8acd4f1b1c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.043852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159bd9a-2b2c-495f-baf7-fd185be6b3d2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.054513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.055213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.055495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.055771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.056572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:59.056888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.058315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.073436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 82/157] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:48:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:59.075420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:59.114975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.116432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.116864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.117287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/users/59a143c30197428cbbf14e9986b00bf7/roles/c735b1c76f774a6f93db499dca9ac387` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.122101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, user_id=59a143c30197428cbbf14e9986b00bf7, role_id=c735b1c76f774a6f93db499dca9ac387)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.123795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.148632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 76/158] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:48:58 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/users/59a143c30197428cbbf14e9986b00bf7/roles/c735b1c76f774a6f93db499dca9ac387 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:59.149733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:59.267470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89e0ee-7704-49f0-92fd-d188804115cc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.315270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.316463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.316950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.317400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.317772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3ba6-2281-42f6-8d19-256481754f76 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.319040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.320648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9c653-3c31-46d4-9597-4ed921bc2a97 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.323329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 83/159] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:59.324766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08c5c8-09f0-4ebf-90c7-60937344c833 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.325248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08c5c8-09f0-4ebf-90c7-60937344c833 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.325725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08c5c8-09f0-4ebf-90c7-60937344c833 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.327209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 84/160] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:59.333668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.334639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.335095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.356897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:59.357758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.358688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.358984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.359242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.359652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:59.360495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.361950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.364164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 77/161] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:59.364426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:59.444896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 85/162] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:48:59.446097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:59.482538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b165d-c699-4b4e-a0bb-cb7651f039ee None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.521866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.522490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.522775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.523050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.523899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:59.524212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.525760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f6a5f-8c0e-4826-887a-778d9234c466 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.540272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 78/163] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:59.541730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59496c-bc50-4e23-ac3e-3b1cd22ac064 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.542017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59496c-bc50-4e23-ac3e-3b1cd22ac064 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.542296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59496c-bc50-4e23-ac3e-3b1cd22ac064 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.547039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 79/164] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:48:59.547039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59496c-bc50-4e23-ac3e-3b1cd22ac064 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:59.604678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a9f6d-07e5-43d1-8e37-c89f589eb129 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.676810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59496c-bc50-4e23-ac3e-3b1cd22ac064 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:59.689573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.690241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.690548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.690822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.748539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:59.749360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.749636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.749908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.773438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e596e0ec-3a79-4a85-bef9-5d5193328db7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:48:59.781355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:48:59.782283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:48:59.783129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:48:59.783743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 80/165] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:48:59.784405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:59.784689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:59.786610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:59.824659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:59.825421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:59.827232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:59.829936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:59.915994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 81/166] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:48:59.917243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.030622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517498e-847d-48a7-b370-ce2c4fabb6e7 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.053340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 86/167] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:00.054189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.066478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.067156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.067437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.067709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.068456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] RBAC: Adding query filter params (name=member) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:00.068760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.070211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.079534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 82/168] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:48:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.080203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.179151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe197c-3557-42e4-b043-672d1ec63af7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.198379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631b2ea-f631-41d1-96b3-462244e61de7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.214964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.215911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.216194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.216467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.217811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.219309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.221046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 83/169] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.221803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.257350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.258154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.258453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.258769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.275774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:00.289177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:00.290054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 87/170] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:00.292408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.325279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1261-3b19-4f53-b281-9b82df4b1244 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.358594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.359530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.359810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.360083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] PATH_INFO: `/v3/users/9b05b527bfb04ddb84873f9672ebcae0` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.361358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] RBAC: Authorizing `identity:get_user(user_id=9b05b527bfb04ddb84873f9672ebcae0)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.362916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.364599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 84/171] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/users/9b05b527bfb04ddb84873f9672ebcae0 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.365608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.447539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750609-b296-4fc9-959f-c4e78fe87c5f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.467083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a0067-f315-4f4f-95dc-e9c4b59bf847 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.501524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.502191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.502497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.502770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.503511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:00.503870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.504414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.505070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.505298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.505462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.505607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.506429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:00.506741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.508162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.513953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 85/172] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.514560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.523107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 88/173] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.523816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.617758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7073f-c946-4413-9973-7dd3639e276b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.661416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.662096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.662380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.662712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.797550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fed16-1108-4223-b942-71aaea8cd3e2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:00.809717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.811257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55939f-cc93-427c-941e-04337e643d16 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.883311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:00.884184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.884473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.884809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] PATH_INFO: `/v3/users/ce068d36274d4c2ab89128932ddb806d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.886038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] RBAC: Authorizing `identity:get_user(user_id=ce068d36274d4c2ab89128932ddb806d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:00.887520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254bb9-3217-4e83-8f1c-b6cc5e17f188 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:00.889354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 89/174] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/users/ce068d36274d4c2ab89128932ddb806d => generated 247 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:00.894749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9e716-a2a1-4173-87cd-b27caa1b1d62 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.895082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9e716-a2a1-4173-87cd-b27caa1b1d62 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.895362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9e716-a2a1-4173-87cd-b27caa1b1d62 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.896915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 90/175] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:00.897615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9e716-a2a1-4173-87cd-b27caa1b1d62 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:00.932457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 86/176] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:00.933868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:00.934112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:00.934408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:00.958849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:00.960018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:01.087633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 87/177] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:01.089010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.102545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9e716-a2a1-4173-87cd-b27caa1b1d62 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.163212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.163830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.164110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.164381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.244369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.245851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.262899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c75de-1ea0-4dce-b8ba-92744149cdc3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.295962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782b68-418f-448e-971a-3947fec0407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.296833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782b68-418f-448e-971a-3947fec0407b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.297115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782b68-418f-448e-971a-3947fec0407b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.297393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62782b68-418f-448e-971a-3947fec0407b None admin] PATH_INFO: `/v3/projects/b469217e345f4099b257694853fed685` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.298603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62782b68-418f-448e-971a-3947fec0407b None admin] RBAC: Authorizing `identity:get_project(project_id=b469217e345f4099b257694853fed685)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.300118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62782b68-418f-448e-971a-3947fec0407b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.302243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 88/178] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/projects/b469217e345f4099b257694853fed685 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:01.303102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782b68-418f-448e-971a-3947fec0407b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.346707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 91/179] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:01.347544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.454460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec86d-7db2-4209-b8ba-4f743bb22ba4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.496683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62782b68-418f-448e-971a-3947fec0407b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.498743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.499402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.499673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.499939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.500766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=b469217e345f4099b257694853fed685, user.id=9b05b527bfb04ddb84873f9672ebcae0) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:01.501069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.502539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.514569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 92/180] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/role_assignments?user.id=9b05b527bfb04ddb84873f9672ebcae0&scope.project.id=b469217e345f4099b257694853fed685&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:01.515447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.557800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.558682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.559009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.559282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.571339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:01.581861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:01.582833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:01.583635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 89/181] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:01.592051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.705467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24a78a-c803-4ab9-95e2-5c4e28e64b14 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.744707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.745962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.746346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.746750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.747744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:01.748280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.749922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.760503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 90/182] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:01.766740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.768093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c2037-3585-4dfd-bf4b-53b5829c9521 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.864220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.865597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.866024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.866460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.867837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.870014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.875369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 93/183] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:01.886024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ef1d6-1d1b-475b-9617-7453046ceed4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:01.895646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:01.928107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:01.929202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:01.929587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:01.933512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:01.953329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:01.955257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:01.957826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 91/184] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:01.964332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.123131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514c408-3586-4f7f-b994-156e3720f2dc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.149873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7be51-4e41-4100-93f3-13d62a134f85 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.162433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.163104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.163383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.163654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.164488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=ce068d36274d4c2ab89128932ddb806d) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:02.164799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.166285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621253f7-33bb-4aab-9cf6-59a0ca96b093 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 94/185] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/role_assignments?user.id=ce068d36274d4c2ab89128932ddb806d&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d41fc-2f00-4005-90f9-8c23d3c26b34 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d41fc-2f00-4005-90f9-8c23d3c26b34 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d41fc-2f00-4005-90f9-8c23d3c26b34 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 95/186] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:02.177815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37c8e-9d44-4cf1-9f94-9a3b2bd9f64c None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.178292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37c8e-9d44-4cf1-9f94-9a3b2bd9f64c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.178292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37c8e-9d44-4cf1-9f94-9a3b2bd9f64c None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.178292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 96/187] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:02.184156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33ebd-c41e-449a-a701-add30b5bfbae None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.184489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33ebd-c41e-449a-a701-add30b5bfbae None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.184722 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33ebd-c41e-449a-a701-add30b5bfbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.194045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.194766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.195051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.195332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.202990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-14b33ebd-c41e-449a-a701-add30b5bfbae None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:02.204201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-14b33ebd-c41e-449a-a701-add30b5bfbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:02.287739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.289529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.291754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 97/188] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:02.293103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.293391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.293665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.314990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:02.316201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:02.402603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 98/189] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:02.403480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.441715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 92/190] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:02.443686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.518615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bb99b-50f3-4f47-a54b-d4ab3e069c66 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.551424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1a426-6332-467b-82ef-5caa2056a52c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.551899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.552443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.552726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.553001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.563115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:02.571168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:02.572010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:02.573392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 99/191] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:02.574066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.588830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.589615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.589888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.590154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.591334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.592839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51b8d4-fc04-472d-ae49-de525eecd1b9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.594450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 93/192] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:02.595610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe642-c66e-4602-a5f0-099a76991d49 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.595921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe642-c66e-4602-a5f0-099a76991d49 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.596194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe642-c66e-4602-a5f0-099a76991d49 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.600856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 94/193] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:02.600856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe642-c66e-4602-a5f0-099a76991d49 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.669301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87393daf-a4a7-4b46-96bb-baa2c375e887 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.690948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe642-c66e-4602-a5f0-099a76991d49 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.700154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.701012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.701294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.701566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.702770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.704223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.705884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 100/194] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:02.706638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.723221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.723875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.724155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.724424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.725161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:02.725458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.726846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6e1b8-8d3b-469e-b518-5f06e4ece7aa None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.734382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 95/195] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:02.735391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.735669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.735940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.752003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:02.753163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:02.808054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440d80e-6df5-4eec-bec3-91059430007a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.813825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 96/196] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:02.815212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.840495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.841163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.841445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.841719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.842447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.843878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.861761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 101/197] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:02 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:02.862683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.928521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a542f68-374a-4326-86b9-8d3ad32eb7c9 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:02.964357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:02.965053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:02.965342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:02.965620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:02.966495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=4c1520328285426292d433237efc53cc, user.id=a3d2bf47d07946e193a4e5b8b5ff1333) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:02.966812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:02.968283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:02.974156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 97/198] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/role_assignments?user.id=a3d2bf47d07946e193a4e5b8b5ff1333&scope.project.id=4c1520328285426292d433237efc53cc&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:02.975017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:02.993300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91606a89-33ef-409c-b51d-c78f7b3079ac None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.044673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.046257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.046257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.046257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.047802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.049400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.051278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 102/199] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:03.057373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.121595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18ba9b-5daf-47e9-88e4-82079fc85bef None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.171443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.172260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.172544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.172818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.189415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:03.198929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:03.200025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:03.201005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 98/200] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:03.205859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.211146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe20124-b64e-4b84-b2ba-f3a294a36eda None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.254510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.255176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.255444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.255721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.325222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220e7b0-c01e-4eec-8663-baeb4eb0c81a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.338202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.340042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.361238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.361913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.362197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.362500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.363488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:03.363597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.365040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.374513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 99/201] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:03.383332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.447744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 103/202] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:49:03.453567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.497815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164174-06ed-42f2-83d4-45dfd9e1ce6d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.553961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.554910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.555178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.555455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] PATH_INFO: `/v3/users/59a143c30197428cbbf14e9986b00bf7` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.557278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] RBAC: Authorizing `identity:get_user(user_id=59a143c30197428cbbf14e9986b00bf7)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.558206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fffa4f-63c3-4735-8d0c-9b786ad36805 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.558763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.565449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 100/203] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/users/59a143c30197428cbbf14e9986b00bf7 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:03.566570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.601071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.602370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.602892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.603889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.606884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.609025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.612617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 104/204] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:03.621029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:03.762039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b6fd8-a93b-40a6-84a2-3fcd7d33f303 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.786507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0031-ebf2-46af-a133-51bc7708c2a1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:03.820757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.821426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.821900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.822215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.852747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:03.853380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:03.853464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:03.854219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:03.855092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4c1520328285426292d433237efc53cc, user.id=a3d2bf47d07946e193a4e5b8b5ff1333) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:03.855409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.856881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:03.870187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 105/205] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/role_assignments?user.id=a3d2bf47d07946e193a4e5b8b5ff1333&scope.project.id=4c1520328285426292d433237efc53cc&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:03.943694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:03.945212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:04.088689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 101/206] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:04.094141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:04.248097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a75dd-eff7-4fe4-b256-6a7597cb3447 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:04.289492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:04.290369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:04.290674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:04.291519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:04.292756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:04.294244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1610-f060-4576-b8b3-566767a64d72 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:04.296130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 106/207] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:04 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:04.303472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:04.441298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834941-1cda-453e-8ad8-5d4cfcc11354 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:04.482575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:04.483625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:04.484122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:04.484623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:04.485709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, scope.project.id=1dbd101bd7884fe899e27996317060c5, user.id=59a143c30197428cbbf14e9986b00bf7) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:04.486242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:04.487974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bbc6e-f27c-4285-8beb-1ae4e87bf65b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:04.499330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 102/208] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:04 2022] GET /identity/v3/role_assignments?user.id=59a143c30197428cbbf14e9986b00bf7&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:05.541641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71dc4d-5ee3-4265-983f-c5459d70c18a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.542198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71dc4d-5ee3-4265-983f-c5459d70c18a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.542277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71dc4d-5ee3-4265-983f-c5459d70c18a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.543804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 107/209] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:05.549043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788b8-c2d6-424d-a748-96afd40f7e29 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.549463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788b8-c2d6-424d-a748-96afd40f7e29 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.549651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788b8-c2d6-424d-a748-96afd40f7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.566661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5c4788b8-c2d6-424d-a748-96afd40f7e29 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:05.567847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5c4788b8-c2d6-424d-a748-96afd40f7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.639569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 103/210] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:05.661900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71dc4d-5ee3-4265-983f-c5459d70c18a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.678528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb7cb-90cb-4c64-84e0-a84179a28c3f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.678830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb7cb-90cb-4c64-84e0-a84179a28c3f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.679095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb7cb-90cb-4c64-84e0-a84179a28c3f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.680812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 104/211] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:05.686360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d940b-ab41-4374-9a4a-6980bc0e54ee None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.686687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d940b-ab41-4374-9a4a-6980bc0e54ee None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.686966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d940b-ab41-4374-9a4a-6980bc0e54ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.703585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6d7d940b-ab41-4374-9a4a-6980bc0e54ee None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:05.704763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6d7d940b-ab41-4374-9a4a-6980bc0e54ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.769798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71dc4d-5ee3-4265-983f-c5459d70c18a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.771551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 105/212] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:05.793438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685ac5-8e50-4033-a04f-a0a64f12b065 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.794133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685ac5-8e50-4033-a04f-a0a64f12b065 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.794950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685ac5-8e50-4033-a04f-a0a64f12b065 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.797082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 106/213] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:05.798023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685ac5-8e50-4033-a04f-a0a64f12b065 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.807513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.808311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.808592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.808864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.819314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a331287b-5acd-46ed-b2ce-b4f18594541a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:05.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:05.834278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:05.835364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:05.836391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 108/214] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:05.838016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.838554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.838913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.856488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:05.857788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.898835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685ac5-8e50-4033-a04f-a0a64f12b065 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.928920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 109/215] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:05.929798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.937388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.938482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.939030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.939500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.940921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:05.942603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:05.948894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 107/216] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:05.950051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.029192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cac9b-deac-4ef0-9f3f-997a6765356a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.045114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33488f07-66df-426f-adb8-7f359ee01a8a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.062168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.062895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.063176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.063446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.064205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:06.064509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.065908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.073615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 110/217] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:06.074406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.084922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.085859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.086340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.086785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.088207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.089779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.115583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 108/218] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:05 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:06.116926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.209970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337d830-0b8a-4cf2-b2fa-f2043960f984 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.248553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.249719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.250205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.250701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.250986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa0f4f-da2c-4330-8197-5353eaeda566 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.263248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:06.271668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:06.272689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:06.273472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:06.276627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 111/219] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:06.282094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.295629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.296604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.296980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.297353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.298693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.300896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.302890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 109/220] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:06.308022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.401159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa510a-a9a8-4bba-8638-a34bacd23d98 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.424297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462f6d6-4ded-41f4-a8a7-5d7df3522f0c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.447854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.448520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.448798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.449540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.450581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:06.450892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.452736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d18ca5-d6ec-4cf4-8174-6f61976465ee None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.464708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 112/221] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:06.466022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.466149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571bf16-d9f8-4976-b3b7-b56a20ff8bff None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.466415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571bf16-d9f8-4976-b3b7-b56a20ff8bff None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.466792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571bf16-d9f8-4976-b3b7-b56a20ff8bff None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.466974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.467241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.467518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.468169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 113/222] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:06.468776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.470351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.473728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.474068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.474296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.476367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 110/223] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:06.477522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.495311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:06.496865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.573890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 114/224] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:06.574696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.585333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f4342-b682-4d6d-a8f7-68b51f34b00c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.619244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.620124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.620407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.620683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] PATH_INFO: `/v3/users/db8464139a4a48d99adce13d54367380` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.635137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] RBAC: Authorizing `identity:get_user(user_id=db8464139a4a48d99adce13d54367380)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.636733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.638908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 111/225] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/users/db8464139a4a48d99adce13d54367380 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:06.639678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.681497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9fd96-8b64-4858-8e15-85bf48af5d65 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.717854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.718770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.719052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.719322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/users/a3d2bf47d07946e193a4e5b8b5ff1333/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.721488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, user_id=a3d2bf47d07946e193a4e5b8b5ff1333, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.722969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.737906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 115/226] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:06 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/users/a3d2bf47d07946e193a4e5b8b5ff1333/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:06.738863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.739879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450666e9-075f-4a10-8ec2-a88569443efb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.779815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.781171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.781467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.781743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.794541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:06.803247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:06.805149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:06.806140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 112/227] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:06.811244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.845136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f7ebc-d391-4d91-9e07-63a984d5aeb0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.882597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.883285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.883558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.883830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.942872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a55aa-b908-40c2-a147-d7c0f4313581 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.976655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.976857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.977376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.977657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.977927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.978753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:06.979056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.979802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.980614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.989856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 113/228] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:06.999362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.127907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 116/229] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:07.136182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.138323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334f14a-2ce4-4ea0-b4fc-3a81f2fd90ff None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.178919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.179811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.180240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.180523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.182308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.183786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.185764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 114/230] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:07.195486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.233799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97eb11-50ef-4d62-99c9-e3c2d3271ee9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.268742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.269609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.269886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.270163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.271390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.272940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.275119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 117/231] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:07 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:07.280835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.352966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589b37-b230-4d2c-8dd2-6ead5aefe28f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.390627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.391811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.392142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.392417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.393654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.395383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.397422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 115/232] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:07 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:07.404715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.500160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d59e4-0cc4-4c0c-b1fb-91d4abf54795 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.550289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da262c-bc30-47a8-b9b9-2bdf62bd7152 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.563999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.564684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.564967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.565241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.566297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=4c1520328285426292d433237efc53cc, user.id=db8464139a4a48d99adce13d54367380) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:07.566426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.567841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db8d99-1e97-418e-ba3c-2c4e20c8a19f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.573749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 118/233] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:07 2022] GET /identity/v3/role_assignments?user.id=db8464139a4a48d99adce13d54367380&scope.project.id=4c1520328285426292d433237efc53cc&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:07.586731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.589016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.589494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.589963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/users/a3d2bf47d07946e193a4e5b8b5ff1333/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.592676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, user_id=a3d2bf47d07946e193a4e5b8b5ff1333, role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.594676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5109c-b473-462f-a26c-af8de66429d1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.613981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 116/234] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:07 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/users/a3d2bf47d07946e193a4e5b8b5ff1333/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:08.807888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0b165-e4a5-46b4-afe7-e2f61d13579a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.808447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0b165-e4a5-46b4-afe7-e2f61d13579a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.808447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0b165-e4a5-46b4-afe7-e2f61d13579a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.810054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 119/235] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:08.815118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f249-0393-4952-b7a6-391a1ccee74b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.815510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f249-0393-4952-b7a6-391a1ccee74b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.815686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f249-0393-4952-b7a6-391a1ccee74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.832662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8627f249-0393-4952-b7a6-391a1ccee74b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:08.833836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8627f249-0393-4952-b7a6-391a1ccee74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:08.914480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 117/236] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:08.950959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0b165-e4a5-46b4-afe7-e2f61d13579a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:08.958431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb40431-19b1-4f78-a33d-878ff515a2ed None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.958958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb40431-19b1-4f78-a33d-878ff515a2ed None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.959446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb40431-19b1-4f78-a33d-878ff515a2ed None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.961255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 118/237] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:08.966985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.967508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.967979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.984492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:08.985943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:09.054537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 119/238] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:09.083505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.148773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0b165-e4a5-46b4-afe7-e2f61d13579a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.222350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.222958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e033-cacb-4ca1-95c0-8695f9bcc3b1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.223187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.223477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.223750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.245732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:09.253617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:09.254525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:09.255342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:09.260603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 120/239] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:09.261628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.261840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.262947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.263523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.263989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.265359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.267133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.269280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 120/240] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:09.274639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.374301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5dcd-04c1-4f62-a7ba-e8a139db0e6a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.392841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264124-3767-471f-a4a0-ea9cda33aea6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.408601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.409614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.410242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.410739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.411634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.413256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.434878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 121/241] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:09 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:09.442496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.442496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.442496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.442496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.442897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] RBAC: Adding query filter params (name=reader) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:09.442897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.442897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.460187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 121/242] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:09.466504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.594629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645cfe6-6ea2-4c3b-b5d8-874b58b9aef7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.629575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.630880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.631354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.631817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] PATH_INFO: `/v3/users/c1540d6f5b754fbead131d11338e2178` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.647014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] RBAC: Authorizing `identity:get_user(user_id=c1540d6f5b754fbead131d11338e2178)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.648887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54681e-82a7-4314-aaf6-cfb10b675a49 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.651793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 122/243] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/users/c1540d6f5b754fbead131d11338e2178 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:09.658440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.676198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6642-615d-45e5-9eac-397c19bd688e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.676839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6642-615d-45e5-9eac-397c19bd688e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.677320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6642-615d-45e5-9eac-397c19bd688e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.678928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 123/244] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:09.684584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.685128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.685594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.706204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:09.707824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:09.815271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 124/245] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:09.828939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac374bf-5cf7-417a-a6db-d6e9c87b6eca None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.840487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.864188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.865920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.866383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.941155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1d971-882c-4b9e-8ab4-66546b1f5d38 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.950223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.952024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.975544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.976380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.976627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.976898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.987239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:09.995876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:09.996775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:09.997578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 125/246] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:10.003144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.057117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 122/247] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:10.063723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.114598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afac21a-6742-424a-b18a-151c7186c5c4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.158151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.159232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.159708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.160173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.161116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:10.162058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.164064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.176444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 126/248] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:10.187432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.200493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed4ba-d45b-4f37-87f2-7163a5dfc148 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.252039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.253291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.253778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.254244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.259137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.260874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.262842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 123/249] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:10.269368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.299847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b82f2-b5ae-43ab-b88c-55c5cc3c040c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.337453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.338377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.338681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.338971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.340167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.341623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289360e-488b-4092-9f4f-8cecc440bf32 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.343577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 127/250] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:10.344786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbff2d-8384-4884-9e3a-40c426ebeb43 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.345064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbff2d-8384-4884-9e3a-40c426ebeb43 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.345337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbff2d-8384-4884-9e3a-40c426ebeb43 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.346710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 128/251] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:10.350321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbff2d-8384-4884-9e3a-40c426ebeb43 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.406238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852fa8a-d3c9-48e1-8410-12c124fbc558 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.452510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.453219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.453571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.453939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.455088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, scope.project.id=4c1520328285426292d433237efc53cc, user.id=c1540d6f5b754fbead131d11338e2178) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:10.455469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.457754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e55a4-df52-482a-9a2b-650ef11eaeb0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.470427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 124/252] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/role_assignments?user.id=c1540d6f5b754fbead131d11338e2178&scope.project.id=4c1520328285426292d433237efc53cc&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:10.470427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.470427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.470427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.472865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbff2d-8384-4884-9e3a-40c426ebeb43 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.486232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:10.487524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:10.540602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.542176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.542682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.543178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.568751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 125/253] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:10.600779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.628774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.630859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbbeed-9d4c-416f-97ed-c4ae10d943ed None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.736261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 129/254] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:10.737939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6bcb-82d3-4627-9232-f7b7d44fba31 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.738449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6bcb-82d3-4627-9232-f7b7d44fba31 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.738930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6bcb-82d3-4627-9232-f7b7d44fba31 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.740910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 130/255] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:10.752007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.752672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.753190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.761674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7453b64-d9bf-41fc-a03b-93e5106c3269 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.779454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:10.781020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:10.797554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.798360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.798660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.798934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.809439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:10.817257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:10.818240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 126/256] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:10.819590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.876084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 131/257] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:10.876972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.931089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fd560-f38d-4f95-95d3-b5a1e49cb2de None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.971238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.972107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.972386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.972660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.973854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.975755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.977769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 127/258] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:10.978529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.085829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3554b-8107-4ab8-9fd8-710eea51423e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.131547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3338d2-8e36-4227-aedb-1bc0f914803a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.168995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.169911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.170199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.170503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.176452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.177353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.177777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.178210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.179127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] RBAC: Adding query filter params (name=member) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:11.179573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.181070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.181696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:11.190303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:11.191402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:11.192418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:11.194732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 128/259] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:11.195588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.205850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 132/260] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.206920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.328633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165cf70-5f74-43ef-a164-98b11790fcfa None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.362588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.363263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.363545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.363817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.364564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:11.364868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.366528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.375332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 133/261] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.376254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.430566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31538ad6-3d73-47ea-ae2f-eb385ba30a53 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.479050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db2221-bfcd-4b2d-9c7f-80d8bd9d706c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.511723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.512601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.512882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.513156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] PATH_INFO: `/v3/users/db8464139a4a48d99adce13d54367380` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.513805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.514337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] RBAC: Authorizing `identity:get_user(user_id=db8464139a4a48d99adce13d54367380)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.514952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.515427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.515867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.516082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.517091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=4c1520328285426292d433237efc53cc, user.id=a3d2bf47d07946e193a4e5b8b5ff1333) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:11.517618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.519280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.526410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 134/262] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/users/db8464139a4a48d99adce13d54367380 => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.526410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.536618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 129/263] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/role_assignments?user.id=a3d2bf47d07946e193a4e5b8b5ff1333&scope.project.id=4c1520328285426292d433237efc53cc&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.537727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.629000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199c89-074a-4614-bc90-7ec4e48e03b2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.663052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.663933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.664215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.664489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] PATH_INFO: `/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.666431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d2bf47d07946e193a4e5b8b5ff1333)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.667745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.669404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 135/264] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/users/a3d2bf47d07946e193a4e5b8b5ff1333 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.675282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.769718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8d800-e965-4935-9213-410554f871e7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.779593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1ce63-51c7-4500-9389-c836560907d9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.814219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.814900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.815182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.815458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.831867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.832745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.833021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.833291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.839107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.840605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.842415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 130/265] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:11.848044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.894326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.895769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.997654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 136/266] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:49:12.005060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:12.010164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc465-f5b1-4eb1-9607-38618bbbbd53 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:12.074337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:12.076236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:12.076526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:12.076806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/users/db8464139a4a48d99adce13d54367380/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:12.081504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, user_id=db8464139a4a48d99adce13d54367380, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:12.083009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:12.108058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 131/267] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:11 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/users/db8464139a4a48d99adce13d54367380/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:49:12.142432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a9a00-a1af-4c4f-9857-54471938ff1d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:12.178472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:12.179270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:12.179550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:12.179827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:12.181076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:12.183124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188374e-34b3-409f-8d88-3977879cac2b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:12.184886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 137/268] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:12 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:12.191092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:12.308121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18da44-903d-4c31-8c32-2b6e320293af None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:12.343312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:12.343976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:12.344260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:12.344536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:12.345380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4c1520328285426292d433237efc53cc, user.id=a3d2bf47d07946e193a4e5b8b5ff1333) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:12.345807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:12.347304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11817e-1701-47e8-b330-9f8c95481aa3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:12.354090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 132/269] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:12 2022] GET /identity/v3/role_assignments?user.id=a3d2bf47d07946e193a4e5b8b5ff1333&scope.project.id=4c1520328285426292d433237efc53cc&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:12.707192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598dc2-54ee-4bc4-9c32-f2dde1bccfbc None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:12.707562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598dc2-54ee-4bc4-9c32-f2dde1bccfbc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:12.707741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20598dc2-54ee-4bc4-9c32-f2dde1bccfbc None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:12.709343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 138/270] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:12.714679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:12.714969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:12.715248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:12.734120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:12.735348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:12.822579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 133/271] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:12.845717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20598dc2-54ee-4bc4-9c32-f2dde1bccfbc None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:12.973003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20598dc2-54ee-4bc4-9c32-f2dde1bccfbc None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.007553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.008363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.008645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.008922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.020835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:13.029461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:13.030469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:13.031284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 139/272] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:13.039530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.141478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac43b8-686d-42f0-a9fe-344dd07c6e3a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.188472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.189148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.189429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.189703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.190484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:13.190793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.192222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a762f3-d695-44bf-99f3-9d05089806c7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.201766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 134/273] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:13.207062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.224057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee86-33f9-43b3-ba71-95a3f486be95 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.224332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee86-33f9-43b3-ba71-95a3f486be95 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.225673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee86-33f9-43b3-ba71-95a3f486be95 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.226140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 135/274] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:13.231436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.231720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.231999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.248873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:13.250043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:13.312081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e5ec1-a6a7-426a-abca-ebde49f4fa7f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.322705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 136/275] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:13.340244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.346898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.348160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.348595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.348867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] PATH_INFO: `/v3/users/018da74e7ab041af931605cff4d70db2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.361616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] RBAC: Authorizing `identity:get_user(user_id=018da74e7ab041af931605cff4d70db2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.363124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b46192-b473-479e-ab53-e6c4786d6a57 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.364906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 140/276] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/users/018da74e7ab041af931605cff4d70db2 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:13.366196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4c68-fde4-4f36-8899-84debe36a8ed None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.366676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4c68-fde4-4f36-8899-84debe36a8ed None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.366778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4c68-fde4-4f36-8899-84debe36a8ed None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.368110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 141/277] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:13.370610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad4c68-fde4-4f36-8899-84debe36a8ed None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.437550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff00a-5a3e-4450-92a3-f7e660709f54 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.467001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad4c68-fde4-4f36-8899-84debe36a8ed None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.470614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.471405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.471683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.471954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.473200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.474744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289a68e-cf00-449a-a3fc-25f170402468 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.476443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 137/278] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:13.477401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.477681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.477951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.495934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:13.497095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:13.504298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.504954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.505233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.505503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.506313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, user.id=018da74e7ab041af931605cff4d70db2, scope.system=all) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:13.506654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.508072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.514190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 142/279] 10.222.0.83 () {56 vars in 1308 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/role_assignments?user.id=018da74e7ab041af931605cff4d70db2&scope.system=all&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:13.514845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.585797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 138/280] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:13.607755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.647554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0990-9e0d-4bad-80f0-5c6fb6a32e17 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.685267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.686315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.686926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.687391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.688314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.689983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.716838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 143/281] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:13 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:13.735492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb9601-1ee2-42e8-9473-c71ffa69f01c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.780202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.781076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.781357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.781625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.794070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:13.802467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:13.803458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:13.804727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 139/282] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:13.810737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:13.924874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39863056-18b0-4588-ab3a-712d9ef45ab3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:13.957790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:13.958488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:13.958773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:13.959049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:13.959792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:13.960096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:13.961470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:13.970134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 144/283] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:13.975670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:14.078186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2651774-4d15-43e0-9b0c-d24a3d780db2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:14.112452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:14.113336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:14.113616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:14.113889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] PATH_INFO: `/v3/users/c1540d6f5b754fbead131d11338e2178` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:14.115112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] RBAC: Authorizing `identity:get_user(user_id=c1540d6f5b754fbead131d11338e2178)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:14.116834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:14.119342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 140/284] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:13 2022] GET /identity/v3/users/c1540d6f5b754fbead131d11338e2178 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:14.124708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:14.227962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c5cd4-f155-4fcf-a3a2-f99b5c21c39d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:14.266357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:14.267263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:14.267553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:14.267828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:14.269019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:14.270491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af016a76-97a2-4623-a285-507e9ce3a6f1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:14.272491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 145/285] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:14 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:14.278105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:14.378471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a9ba-a818-4421-8dcc-f34c334076af None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:14.412119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:14.413001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:14.413285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:14.413559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/users/c1540d6f5b754fbead131d11338e2178/roles/c735b1c76f774a6f93db499dca9ac387` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:14.415577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, user_id=c1540d6f5b754fbead131d11338e2178, role_id=c735b1c76f774a6f93db499dca9ac387)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:14.417544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5700436-3e29-4b9a-b1da-eb101559c1e5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:14.435588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 141/286] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:14 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/users/c1540d6f5b754fbead131d11338e2178/roles/c735b1c76f774a6f93db499dca9ac387 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:14.806743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9c27a-4198-4d2e-a483-5a43bd8050f0 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:14.807081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9c27a-4198-4d2e-a483-5a43bd8050f0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:14.807246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9c27a-4198-4d2e-a483-5a43bd8050f0 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:14.808894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 146/287] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:14.816170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:14.816170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:14.816170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:14.838433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:14.839970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:14.942234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 142/288] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:14.980736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9c27a-4198-4d2e-a483-5a43bd8050f0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:15.119098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9c27a-4198-4d2e-a483-5a43bd8050f0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:15.162740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:15.163542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:15.163828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:15.164099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:15.183269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:15.191499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:15.192385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:15.193192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 147/289] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:15.198569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:15.332393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5b8f-374a-4fc4-a007-2174064df9c4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:15.380257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:15.380860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:15.381140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:15.381415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:15.382156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:15.382485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:15.383858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:15.397717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 143/290] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:15.403793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:15.526295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01379a-d3b4-409d-bd71-c32e79ebba27 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:15.565770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:15.567038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:15.567543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:15.568013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] PATH_INFO: `/v3/users/db8464139a4a48d99adce13d54367380` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:15.569407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] RBAC: Authorizing `identity:get_user(user_id=db8464139a4a48d99adce13d54367380)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:15.571089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:15.573743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 148/291] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:15 2022] GET /identity/v3/users/db8464139a4a48d99adce13d54367380 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:15.579554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:15.738676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acc9a1-8664-4d39-88c5-0120f7464470 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:15.780085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:15.780733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:15.781016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:15.781293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:15.873027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:15.874583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:16.033490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 144/292] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:16.039614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.175450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700065e-be2b-4680-af3f-d6accb1722b8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:16.217817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:16.219116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.219610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.220081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.221480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:16.223136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b61be9-90b8-428d-8b71-e92d0e89550e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:16.225364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 149/293] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:16.235483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.347964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070117-b959-411a-a12c-13a544085305 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.348600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070117-b959-411a-a12c-13a544085305 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.349107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070117-b959-411a-a12c-13a544085305 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.350956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 150/294] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:16.356110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6f93-6fa2-4fd4-8437-98f745e75e90 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.356300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672c7a8-9429-4ea2-9986-cf518d377b9a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:16.356398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6f93-6fa2-4fd4-8437-98f745e75e90 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.356657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6f93-6fa2-4fd4-8437-98f745e75e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.386718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-440d6f93-6fa2-4fd4-8437-98f745e75e90 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:16.387920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-440d6f93-6fa2-4fd4-8437-98f745e75e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:16.390559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:16.391217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.391497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.391767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.392696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=4c1520328285426292d433237efc53cc, user.id=db8464139a4a48d99adce13d54367380) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:16.393012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:16.394459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:16.400699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 145/295] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/role_assignments?user.id=db8464139a4a48d99adce13d54367380&scope.project.id=4c1520328285426292d433237efc53cc&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:16.524263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 151/296] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:16.559000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.693497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfba32c-7908-40b0-aae0-b796a58c2e76 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:16.699547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed50c-8f7d-41d4-b55e-3bb0bb070ef3 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.699837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed50c-8f7d-41d4-b55e-3bb0bb070ef3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.700112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed50c-8f7d-41d4-b55e-3bb0bb070ef3 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.701762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 152/297] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:16.711598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e1e0-5edf-4c28-b701-8a4d39ec05dd None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.711891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e1e0-5edf-4c28-b701-8a4d39ec05dd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.712159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e1e0-5edf-4c28-b701-8a4d39ec05dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.730346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:16.731190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.731523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.731802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.742639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:16.744737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5694e1e0-5edf-4c28-b701-8a4d39ec05dd None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:16.745965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5694e1e0-5edf-4c28-b701-8a4d39ec05dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:16.750409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:16.751295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:16.752080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 146/298] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:16.760908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.846132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 153/299] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:16.847726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e6d8-f4ca-4a30-8ea5-d8e9e87c959c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.848104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e6d8-f4ca-4a30-8ea5-d8e9e87c959c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.848454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e6d8-f4ca-4a30-8ea5-d8e9e87c959c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.850053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 154/300] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:16.856593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.856879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.857156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.875091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:16.876344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:16.880083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f516b8-37c7-44b3-8509-dd6e5d0753e4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:16.919304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:16.920403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.920739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.921345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.922098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:16.922429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:16.923834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:16.932634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 147/301] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:16.933645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.948259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 155/302] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:16.949069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.043423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f796e9e-bc71-4d8d-b5cc-6b8a75f05fd6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.048694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22f0f-6421-49c1-991e-e659c4efe20b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.075426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.076215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.076492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.076760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.077895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.079380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.080952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 148/303] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:17.081696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.087855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.088752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.089035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.089309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] PATH_INFO: `/v3/users/018da74e7ab041af931605cff4d70db2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.090545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] RBAC: Authorizing `identity:get_user(user_id=018da74e7ab041af931605cff4d70db2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.091991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.098081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 156/304] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/users/018da74e7ab041af931605cff4d70db2 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:17.098081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.189107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79dbf1-3ed5-4d2d-8018-23d8fbdf1e3a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.193113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42873845-6faf-43d2-9e9f-53b0873239e6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.223109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.223829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.224109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.224385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.226457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.227114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.227389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.227654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] PATH_INFO: `/v3/groups` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.228822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.230226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b8697-fa1d-4fbc-b07a-072f7674161a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.234510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:17.243125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:17.245044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 149/305] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:17.245846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.246592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 157/306] 10.222.0.83 () {60 vars in 1134 bytes} [Tue May 10 14:49:17 2022] POST /identity/v3/groups => generated 229 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:17.247437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934951-0ba3-448b-8b93-828bf6f8e0fe None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.247437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934951-0ba3-448b-8b93-828bf6f8e0fe None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.247437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934951-0ba3-448b-8b93-828bf6f8e0fe None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.248699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 158/307] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:17.250482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934951-0ba3-448b-8b93-828bf6f8e0fe None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.354695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934951-0ba3-448b-8b93-828bf6f8e0fe None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.364303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6617-3b3a-4cbf-83e8-a6a43e156e80 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.388465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.389084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.389365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.389637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.390429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:17.390716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.392088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f8da0-761c-454c-b6b1-fcaf424297dd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.398568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.399454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.399734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.400530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] PATH_INFO: `/v3/system/users/018da74e7ab041af931605cff4d70db2/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.402068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=018da74e7ab041af931605cff4d70db2, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.402532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 159/308] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:17.403647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.403932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.404203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.408095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.419445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 150/309] 10.222.0.83 () {58 vars in 1334 bytes} [Tue May 10 14:49:17 2022] PUT /identity/v3/system/users/018da74e7ab041af931605cff4d70db2/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:17.426532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.427946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:17.429311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:17.518271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 160/310] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:17.541703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.548170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dba161-7929-43c4-8c4e-65993382c2d8 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.593840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.595011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.595955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.596379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] PATH_INFO: `/v3/users/5f9f1a25a3f248afa24da30f96418012` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.615095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f1a25a3f248afa24da30f96418012)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.616834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.618670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 151/311] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/users/5f9f1a25a3f248afa24da30f96418012 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:17.624392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.653812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576137a1-cc36-4934-935e-fc7f0b18e52c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.688158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.688969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.689257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.689532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.703279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:17.711052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:17.711946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:17.712830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 161/312] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:17.722188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:17.733495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8fb07-2477-442b-a316-a185d7e55d8f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.767646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.768683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.769158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.769638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.770681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, user.id=5f9f1a25a3f248afa24da30f96418012, scope.system=all) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:17.771205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.772817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.778780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 152/313] 10.222.0.83 () {56 vars in 1308 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/role_assignments?user.id=5f9f1a25a3f248afa24da30f96418012&scope.system=all&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:17.839359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860b62-3eca-4c31-b9f3-ac93212317b7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:17.875994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:17.876683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:17.876970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:17.877247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:17.878002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:17.878347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:17.879748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:17.889589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 162/314] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:17.895718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:18.017444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e68c-3a8a-43d8-9e33-301277e63e78 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:18.050853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:18.052062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:18.052550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:18.053018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] PATH_INFO: `/v3/users/c1540d6f5b754fbead131d11338e2178` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:18.054421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=c1540d6f5b754fbead131d11338e2178)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:18.056065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:18.058724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 153/315] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:17 2022] GET /identity/v3/users/c1540d6f5b754fbead131d11338e2178 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:18.064288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:18.167125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0ca49-ee18-4d69-89e9-8108f1927280 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:18.201609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:18.202265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:18.202571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:18.202852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:18.270872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:18.272470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:18.370410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 163/316] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:18.375840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:18.475069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818806b9-c0b6-4a7e-8a8e-ff372051f7c3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:18.512508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:18.513751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:18.514225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:18.514715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:18.516137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:18.517797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01078479-c471-415d-b7e2-82f70effdec0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:18.520024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 154/317] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:18 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:18.525977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:18.627607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0bb2a-7483-4d15-bc71-def0c02b9bf2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:18.664976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:18.666159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:18.666460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:18.666735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:18.667613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, scope.project.id=4c1520328285426292d433237efc53cc, user.id=c1540d6f5b754fbead131d11338e2178) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:18.667918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:18.669377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadb508-cbbe-4ef3-bddf-d274992e2db4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:18.675517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 164/318] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:18 2022] GET /identity/v3/role_assignments?user.id=c1540d6f5b754fbead131d11338e2178&scope.project.id=4c1520328285426292d433237efc53cc&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:19.643186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207d40-5094-44fd-9a20-06dd99e34ff5 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.643709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207d40-5094-44fd-9a20-06dd99e34ff5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.643784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207d40-5094-44fd-9a20-06dd99e34ff5 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.645091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 155/319] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:19.650633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.651045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.651183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.667806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:19.668979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:19.685080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2461c-89ab-401b-9421-4cde028179cd None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.685357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2461c-89ab-401b-9421-4cde028179cd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.685796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2461c-89ab-401b-9421-4cde028179cd None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.687166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 156/320] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:19.692566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.692848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.693301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.710140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:19.711352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:19.734480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 165/321] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:19.757565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:19.780026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 157/322] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:19.802369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:19.860995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598f333-976c-42b3-9e99-9a7991c8833c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:19.894475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:19.895179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.895455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.895724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.896854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:19.898301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:19.900113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 166/323] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:19.905057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e53e-8af7-4f8a-82a0-a22a452cfa2f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:19.905513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:19.944334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:19.945132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:19.945415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:19.945687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:19.955524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:19.963391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:19.964256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:19.965046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:19.969400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 158/324] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:19.970199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.007170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cddbe-e974-42f6-9a4e-3d9de2b5d230 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.039483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.040165 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.040957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.041235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] PATH_INFO: `/v3/groups` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.041925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.043398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74806f84-b4f8-491c-9019-07d90eda0cb7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.056178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 167/325] 10.222.0.83 () {60 vars in 1134 bytes} [Tue May 10 14:49:19 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:49:20.057764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa12a2-5df6-4c25-9444-f1f18da0557f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.058233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa12a2-5df6-4c25-9444-f1f18da0557f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.058426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa12a2-5df6-4c25-9444-f1f18da0557f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.060332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 168/326] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:20.065538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e35d-ec3a-426a-b1be-58f938a18e7f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.066083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.066374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.066690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.083927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:20.085214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:20.100924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.101706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.101991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.102265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.103666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:20.103975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.105363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.114161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 159/327] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:20.121061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.157329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 169/328] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:20.179555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.224977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9133-c759-470f-8a4b-4397a76f1a3d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.265362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.266243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.266568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.267628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] PATH_INFO: `/v3/users/018da74e7ab041af931605cff4d70db2` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.268099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] RBAC: Authorizing `identity:get_user(user_id=018da74e7ab041af931605cff4d70db2)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.269484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.271211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 160/329] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:20 2022] GET /identity/v3/users/018da74e7ab041af931605cff4d70db2 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:20.278168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.283784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19c6e-e7b4-4308-901b-fc5f1f5b9f20 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.322761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.323582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.323866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.324140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.339000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:20.347730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:20.348643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:20.349454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 170/330] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:20.355545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.403711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa4e9-03c9-4cb5-9177-ca8f58f1a559 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.440071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.440964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.441398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.441810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.442819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] RBAC: Adding query filter params (role.id=edf562e307eb48899cd024cddf6d7be3, user.id=018da74e7ab041af931605cff4d70db2, scope.system=all) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:20.443267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.444852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.457375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 161/331] 10.222.0.83 () {56 vars in 1308 bytes} [Tue May 10 14:49:20 2022] GET /identity/v3/role_assignments?user.id=018da74e7ab041af931605cff4d70db2&scope.system=all&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:20.466974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95085d6-5981-487b-b3ce-be2ce523261d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.506033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.506809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.509685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.509685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.509685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:20.509685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.510006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.519201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 171/332] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:20.525362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.647817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d9ec-8e78-4ac2-abb0-e477649c2e29 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.686158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.687089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.687372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.687679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] PATH_INFO: `/v3/users/5f9f1a25a3f248afa24da30f96418012` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.688915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f1a25a3f248afa24da30f96418012)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.690436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032f73-b774-4549-8a0b-c05e21b46b3b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.692393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 162/333] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:20 2022] GET /identity/v3/users/5f9f1a25a3f248afa24da30f96418012 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:20.698203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:20.804775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38a7ef-e5e5-46ee-9fd0-433b5fb18114 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:20.850163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:20.851080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:20.851364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:20.851648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] PATH_INFO: `/v3/system/users/5f9f1a25a3f248afa24da30f96418012/roles/c735b1c76f774a6f93db499dca9ac387` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:20.853160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5f9f1a25a3f248afa24da30f96418012, role_id=c735b1c76f774a6f93db499dca9ac387)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:20.854591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c615c-3ed9-4634-96dc-ed4530ade70f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:20.864462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 172/334] 10.222.0.83 () {58 vars in 1334 bytes} [Tue May 10 14:49:20 2022] PUT /identity/v3/system/users/5f9f1a25a3f248afa24da30f96418012/roles/c735b1c76f774a6f93db499dca9ac387 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:49:22.632243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c974-6870-4acb-b473-6e4340c207cb None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:22.632767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c974-6870-4acb-b473-6e4340c207cb None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:22.632767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c974-6870-4acb-b473-6e4340c207cb None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:22.634974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 163/335] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:22.640491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:22.640924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:22.641126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:22.663557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:22.664750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:22.745189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 173/336] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:22.783287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0c974-6870-4acb-b473-6e4340c207cb None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:22.909035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0c974-6870-4acb-b473-6e4340c207cb None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:22.952271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:22.953066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:22.953343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:22.953609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:22.967608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:22.975323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:22.976203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:22.976989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 164/337] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:22.983055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:23.123017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaea89-ea43-42d2-b1c3-101752c0b22b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:23.171169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:23.171783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.172070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.172350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.173103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] RBAC: Adding query filter params (name=member) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:23.173413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:23.174820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:23.183434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 174/338] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:23.189277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:23.318764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e536ecd-49a8-489c-8820-fd694d3c69ec None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:23.357479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:23.358143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.358454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.358733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.440311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:23.441693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:23.552199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 165/339] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:23.558513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:23.701412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33517-6ece-4b18-901e-0c868648178f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:23.741082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:23.742333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.742837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.743331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.748199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:23.749876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a08a94-3457-473e-b55e-4afb73a3cd52 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:23.752209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 175/340] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:23 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:23.758106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:23.850688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f24c6-8a2a-45f1-80d9-d54260e02afb None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.851342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f24c6-8a2a-45f1-80d9-d54260e02afb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.851842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f24c6-8a2a-45f1-80d9-d54260e02afb None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.853950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 176/341] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:23.866996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fc8-14d0-4423-868f-2c7947f6a43c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.867602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fc8-14d0-4423-868f-2c7947f6a43c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.868172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fc8-14d0-4423-868f-2c7947f6a43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.878829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158222cc-deb7-4278-8c3c-385aafcdd785 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:23.899823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-76e53fc8-14d0-4423-868f-2c7947f6a43c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:23.901460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-76e53fc8-14d0-4423-868f-2c7947f6a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:23.934596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:23.935563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.935846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.936117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.942812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:23.944481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e0f3e-6dcc-4e4b-be0f-0b90c4878d2a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:23.947843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 166/342] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:23 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:23.955878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424048f-85d0-4889-96ff-eacbedc6ddb1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:23.956491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424048f-85d0-4889-96ff-eacbedc6ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:23.956920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424048f-85d0-4889-96ff-eacbedc6ddb1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:23.958947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 167/343] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:23.960075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424048f-85d0-4889-96ff-eacbedc6ddb1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.037953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 177/344] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:24.042116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23847-34c8-40f8-aa74-8965e666de68 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.042675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23847-34c8-40f8-aa74-8965e666de68 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.043169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23847-34c8-40f8-aa74-8965e666de68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.063674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9cd23847-34c8-40f8-aa74-8965e666de68 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:24.065192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9cd23847-34c8-40f8-aa74-8965e666de68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:24.100759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424048f-85d0-4889-96ff-eacbedc6ddb1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.147501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.148420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.148840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.149249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.150247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=1dbd101bd7884fe899e27996317060c5) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:24.150742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:24.152369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa269b49-9bea-440c-8f0e-a4ca24a58032 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:24.154190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 178/345] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:24.155888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbacac-7480-4b47-a36d-ac5e5b1fb0d5 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.156448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbacac-7480-4b47-a36d-ac5e5b1fb0d5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.156924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbacac-7480-4b47-a36d-ac5e5b1fb0d5 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.158472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 179/346] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:24.159348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbacac-7480-4b47-a36d-ac5e5b1fb0d5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.162712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 168/347] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:23 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:24.164978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.165268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.165577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.195415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:24.197866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:24.287875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 169/348] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:24.289616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.304832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbacac-7480-4b47-a36d-ac5e5b1fb0d5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.349675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.350856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.351320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.351748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.366935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:24.378381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:24.379596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:24.380604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:24.383341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 180/349] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:24.384893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.437054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5cd28-2c48-4c97-a755-6bdb570563d5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.478592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.479400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.479681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.479952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.490861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-542ae842-ddd4-4bb2-9d75-4139668ff12a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:24.499648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:24.500752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:24.501719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 170/350] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:24.502461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb85d5-0956-41e9-97f0-9713f68cd27e None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.506867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb85d5-0956-41e9-97f0-9713f68cd27e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.507082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb85d5-0956-41e9-97f0-9713f68cd27e None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.508394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 171/351] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:24.509144 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb85d5-0956-41e9-97f0-9713f68cd27e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.512210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f8b-38c7-45e0-b10b-8ebae86c6b8c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.545984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.547057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.547494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.547914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.560253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:24.569085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:24.570178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:24.571107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:24.573353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 181/352] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:24.574254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.617489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb85d5-0956-41e9-97f0-9713f68cd27e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.652031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.652268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.652391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.652632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.653375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:24.653685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:24.655086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e009-fe16-4481-9bea-632bcf032645 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:24.663518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 172/353] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:24.665074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.665074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.665854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.682858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:24.684038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:24.684950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55443d-83b2-40f7-a42c-bc8010e18f98 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.718326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.719812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.720267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.720735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.721665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] RBAC: Adding query filter params (name=reader) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:24.722142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:24.723834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:24.733589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 182/354] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:24.734696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.756898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 173/355] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:24.757739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.836706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab6836-b737-47b6-8804-31ea68ff1a6f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.856517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f071-2a59-4898-a600-d9dbc3b9bce0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:24.874903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.875837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.876260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.876680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.877965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:24.878447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:24.880478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:24.891066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 183/356] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:24.891924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:24.896889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:24.897909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:24.898382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:24.898893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:24.981684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:24.984499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.015240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89c2a-6521-4cc0-9172-3864f6d36c0c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.061065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.062366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.062892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.063554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] PATH_INFO: `/v3/users/5f9f1a25a3f248afa24da30f96418012` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.065091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f1a25a3f248afa24da30f96418012)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.067823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.071322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 184/357] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/users/5f9f1a25a3f248afa24da30f96418012 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.072450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.111595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 174/358] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:25.112777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.183838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287484c-c561-4c10-9f36-c2b228c2729e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.211244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee191be-1ea0-482b-aedf-42012bdf2fc9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.219695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.220642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.221023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.221390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.232145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:25.240703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:25.241692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:25.242527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:25.243142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 185/359] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:25.243729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.251364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.252318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.252801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.253273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.336346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.338415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.362098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e506-2ac9-4504-823e-4ec6da8d1b25 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.400745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.401773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.402052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.402322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.403203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] RBAC: Adding query filter params (role.id=c735b1c76f774a6f93db499dca9ac387, user.id=5f9f1a25a3f248afa24da30f96418012, scope.system=all) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:25.403513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.404946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.418141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 186/360] 10.222.0.83 () {56 vars in 1308 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/role_assignments?user.id=5f9f1a25a3f248afa24da30f96418012&scope.system=all&role.id=c735b1c76f774a6f93db499dca9ac387 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.419414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.452856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 175/361] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:25.453767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.555064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e844cd-08fb-45a5-8d0f-803ad92ec66c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.571321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e4ba-3f6d-40c3-ae1d-953b6cd5eaac None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.590665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.591556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.591836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.592109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.593377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.594944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231e3ab-cba2-4359-bb61-599ddb7eeb0a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.597039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 187/362] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.598310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1046a1-dace-4c84-8a30-02b35be300cb None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.598621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1046a1-dace-4c84-8a30-02b35be300cb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.598896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1046a1-dace-4c84-8a30-02b35be300cb None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.600393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 188/363] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:25.601107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1046a1-dace-4c84-8a30-02b35be300cb None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.607920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.608606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.608884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.609153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.609910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:25.610215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.611666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.621710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 176/364] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.623770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.711539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1046a1-dace-4c84-8a30-02b35be300cb None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.728442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae26251-2a7c-41b6-8ed2-1d00471d6927 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.747676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.748786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.748786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.749066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.750282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.752797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44905a-6185-4bf4-924d-2e1f40920230 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.754754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 189/365] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.756005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.756293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.756565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.769587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.770891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.771234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.771585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.775209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:25.779121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:25.779921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 177/366] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:25.780624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.788601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:25.789977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:25.860394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 190/367] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:25.861461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:25.916822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f580ff5-ad37-4351-b224-324a25267db2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:25.956384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:25.957028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:25.957305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:25.957576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:25.966873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231e29-6c48-40c6-aeea-55fbed6e2bb6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.005431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.006691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.007178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.007667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.009119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.010804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.013702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 191/368] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:26.014789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.035566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.037055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.140097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7aab-ffb5-4d2b-b833-03b13c63a4a9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.161662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 178/369] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:26.162314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.189598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.190182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.190494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.190771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.191601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=4c1520328285426292d433237efc53cc) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:26.191939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.193344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.199489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 192/370] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=4c1520328285426292d433237efc53cc&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:26.200353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.284280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76113-d316-4a5f-8615-c73ab410fc20 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.312416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b280-bccf-4a67-8e4f-b9fe6e71e91c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.321158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.322026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.322349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.322663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.334156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:26.342935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:26.343905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:26.344707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 179/371] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:26.346039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.359781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.360455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.360733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.361004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.361838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=1dbd101bd7884fe899e27996317060c5) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:26.362151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.363622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.369898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 193/372] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:26.370564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.481433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d911-2e39-4d10-8679-0cf8406cf1bd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.499988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc79c-a986-4db8-9865-8af88220c987 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.518624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.519508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.519792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.520070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.521337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.522862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.529856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 180/373] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:26.531244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.536253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.536929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.537209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.537480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.538223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:26.538660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.539989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.549913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 194/374] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:26.555413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.689759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563136-4fc2-496b-929c-ce11fc035285 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.735903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751eb66-374b-46ff-9212-3448949cb8df None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.754944 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.756180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.756662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.757134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.759792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.761468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.765760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 181/375] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:26.770278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:26.787593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.788933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.789430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.789892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.924313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.926068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:26.942650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed84e-3362-4053-8a62-f8d0fc6cac2b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:26.987573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:26.988300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:26.988662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:26.988989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:26.990073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4c1520328285426292d433237efc53cc) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:26.990734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:26.992622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ad21f-809b-4352-abe6-2ca5a20937a1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.000155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 182/376] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=4c1520328285426292d433237efc53cc&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:27.002037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ab56-d344-4575-8700-98cbea106c1e None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.002426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ab56-d344-4575-8700-98cbea106c1e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.002935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ab56-d344-4575-8700-98cbea106c1e None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.005336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 183/377] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:27.012127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.015650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.016148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.054871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 195/378] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:49:27.062429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:27.062429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:27.063718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.166773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee800d-82af-4846-a62e-f553ecd3ea38 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.209701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.210615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.210898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.211172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] PATH_INFO: `/v3/projects/06d48372060e456ea0c20a517b83a4d3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.213136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] RBAC: Authorizing `identity:get_project(project_id=06d48372060e456ea0c20a517b83a4d3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:27.214635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.220145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 184/379] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:27.220322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 196/380] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/projects/06d48372060e456ea0c20a517b83a4d3 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:27.222751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.247932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.342286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c43f-4b8e-47dd-85e8-f96076e2fedb None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.380221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.381129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.381414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.382217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] PATH_INFO: `/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.387530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] RBAC: Authorizing `identity:get_group(group_id=fe29b868d0bb4e0ca04d58defb4328e8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:27.390140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.392276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 185/381] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:27.397648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.450598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962da3c-c258-4593-86fb-9b951749b479 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.508423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b342524-6e68-4bf4-a9bb-39b53da28d18 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.539087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.539902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.540185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.540491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.544851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.545510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.545790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.546064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.546922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] RBAC: Adding query filter params (group.id=fe29b868d0bb4e0ca04d58defb4328e8, role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=06d48372060e456ea0c20a517b83a4d3) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:27.547544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:27.548999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.555123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 186/382] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/role_assignments?group.id=fe29b868d0bb4e0ca04d58defb4328e8&scope.project.id=06d48372060e456ea0c20a517b83a4d3&role.id=57e23e25b8c046ba882d07965d852386 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:27.562548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:27.570702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:27.573714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:27.575634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 197/383] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:27.582411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.682018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c29b0-a2b0-412a-8fb5-6d5be8027577 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.717331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.718011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.718288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.718588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.719337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:27.719644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:27.721044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.734504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 187/384] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:27.736221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:27.844560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008db4c-e979-44c9-b534-5de21cd790e1 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:27.879956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:27.880846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:27.881132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:27.881409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:27.882692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:27.884719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:27.886725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 198/385] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:27.893702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:28.013745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad87a1-c247-4cf3-aa20-f0a6c2574a8b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:28.054648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:28.055750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:28.056684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:28.057122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:28.063095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:28.064794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bffd-4a23-40ff-9721-2a4d49b5690d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:28.066940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 188/386] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:27 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:28.073199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:28.198825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0997bfc-e300-4b1f-8896-1ffd038ac775 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:28.232020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:28.232905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:28.233185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:28.233455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/groups/d0735f5dc83845e189b80fb5568f715a/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:28.237207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, group_id=d0735f5dc83845e189b80fb5568f715a, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:28.238671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02a789-50c7-424e-8402-3ad739429436 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:28.252391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 199/387] 10.222.0.83 () {58 vars in 1442 bytes} [Tue May 10 14:49:28 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/groups/d0735f5dc83845e189b80fb5568f715a/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:29.016381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d16d3c-32bb-4479-a04e-cee797254eaf None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.017410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d16d3c-32bb-4479-a04e-cee797254eaf None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.017688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d16d3c-32bb-4479-a04e-cee797254eaf None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.019376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 189/388] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:29.025593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b1b6-22ba-4a1c-ab23-33d0e4ae21b6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.026500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b1b6-22ba-4a1c-ab23-33d0e4ae21b6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.026977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b1b6-22ba-4a1c-ab23-33d0e4ae21b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.045647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6161b1b6-22ba-4a1c-ab23-33d0e4ae21b6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:29.047192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6161b1b6-22ba-4a1c-ab23-33d0e4ae21b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:29.119071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 200/389] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:29.139603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d16d3c-32bb-4479-a04e-cee797254eaf None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.258126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00d7a-2d98-48ad-a823-975804333ec2 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.258315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00d7a-2d98-48ad-a823-975804333ec2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.258740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00d7a-2d98-48ad-a823-975804333ec2 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.260895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 201/390] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:29.266646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.266933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.267208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.273393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d16d3c-32bb-4479-a04e-cee797254eaf None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:29.286262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:29.288051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:29.315644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:29.316440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.316720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.316993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.327833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:29.336255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:29.337242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:29.338048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 190/391] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:29.346689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.379000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 202/392] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:29.423315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.526265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb0c36-ea52-4d7f-9640-948c8e455692 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:29.533517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fab632-9520-465e-b50a-de4cca6cf3e6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:29.567753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:29.568805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.569254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.569687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.581080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:29.583082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.585576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.586033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.586980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:29.587491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:29.588473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:29.590030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:29.599997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:29.601154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:29.602053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:29.608851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 203/393] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:29.615357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 191/394] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:29.617576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.621442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.780469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11254e3-2889-4a25-80e6-ec4d79aa53f0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:29.803779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda76dc-0346-4335-b25e-74e7f955c1e7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:29.836688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:29.837433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.837715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.837985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.838760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:29.839065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:29.840422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:29.848752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 204/395] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:29.854834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.856766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:29.857867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:29.858360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:29.858858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:29.863096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:29.864781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:29.867219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 192/396] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:29.872891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:29.969606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbccfe-cfe2-40dc-afa0-71a146726229 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.005674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.006567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.006965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.007424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.009683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.011525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68156a3-e997-46dc-9352-a7c722270fc0 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.012595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce834e-bfc7-4a49-a613-464dda79857e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.013425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 205/397] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:30.018123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1f1a3-d7ae-4214-9024-e1d2ae4726fe None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.018480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1f1a3-d7ae-4214-9024-e1d2ae4726fe None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.018770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1f1a3-d7ae-4214-9024-e1d2ae4726fe None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.020076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 206/398] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:30.022762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1f1a3-d7ae-4214-9024-e1d2ae4726fe None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.055733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.057277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.057525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.059015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.059529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.060994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f428e40-0e69-4fa1-b9e0-30dbd72d94ed None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.063260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 193/399] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:29 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:30.064687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.065001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.065277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.086113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:30.087904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:30.162851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1f1a3-d7ae-4214-9024-e1d2ae4726fe None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.174505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 194/400] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:30.175667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.197561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.198715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.198939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.199213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.200477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.202094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.208658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 207/401] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:30 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:30.209537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.278242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d334ce-2af6-47e4-ba65-f785bb67c5a9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.311277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909e83c-58b1-49d2-92fa-90cc7becc310 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.315813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.317243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.317719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.318179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/groups/d0735f5dc83845e189b80fb5568f715a/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.320765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, group_id=d0735f5dc83845e189b80fb5568f715a, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.323254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.346442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 195/402] 10.222.0.83 () {58 vars in 1442 bytes} [Tue May 10 14:49:30 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/groups/d0735f5dc83845e189b80fb5568f715a/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:30.346442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.362192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.363396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.363881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.364416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.391355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:30.413148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:30.414124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:30.414987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:30.420193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 208/403] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:30.425914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.514736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9fee-50a1-4209-9e01-f1007fb8eefe None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.578724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.580080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.580361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.580632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5/groups/d0735f5dc83845e189b80fb5568f715a/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.584934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd101bd7884fe899e27996317060c5, group_id=d0735f5dc83845e189b80fb5568f715a, role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.586369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.615981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 196/404] 10.222.0.83 () {58 vars in 1442 bytes} [Tue May 10 14:49:30 2022] PUT /identity/v3/projects/1dbd101bd7884fe899e27996317060c5/groups/d0735f5dc83845e189b80fb5568f715a/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:30.621270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce3c0c-ea15-4501-925d-bed16fca3190 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.666966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.667649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.667932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.668202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.668958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:30.669261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.670879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd103fe-050b-4bab-a859-933b22353d95 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.683439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 209/405] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:30.688815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.802920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ad872-a6a7-4258-82e7-bef9945eb725 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.819928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b6ed-3a2e-46fa-899a-f8f7f48f5cf5 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.820252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b6ed-3a2e-46fa-899a-f8f7f48f5cf5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.820683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b6ed-3a2e-46fa-899a-f8f7f48f5cf5 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.822960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 210/406] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:30.837558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.838187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.838690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.844520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:30.845417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:30.845685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:30.845981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:30.847302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:30.849027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:30.851180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 197/407] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:30 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:30.858817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:30.880054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:30.882304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:30.979231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be2b2-4b1d-45b4-857e-d909677cc782 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:30.985240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 211/408] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:31.017063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.017937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.018220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.018528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.019914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.020813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.022159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.025097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 198/409] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:30 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:31.032312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.126502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bb41e-e2d1-4c22-878e-b9107e62d1aa None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.159883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9b381-41b9-4560-bf13-85a97db0f526 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.163351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.164651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.165644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.166111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.176867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:31.185807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:31.186741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:31.187545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:31.190235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 212/410] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:31.199496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.219147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.220068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.220307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.220578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc/groups/d0735f5dc83845e189b80fb5568f715a/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.227256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1520328285426292d433237efc53cc, group_id=d0735f5dc83845e189b80fb5568f715a, role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.229275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91ead-a944-4a3b-81c0-f98dbcd73ad8 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.258010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 199/411] 10.222.0.83 () {58 vars in 1442 bytes} [Tue May 10 14:49:31 2022] PUT /identity/v3/projects/4c1520328285426292d433237efc53cc/groups/d0735f5dc83845e189b80fb5568f715a/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:31.312969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd789-4e32-4fe7-9d2e-a5637e85d1a7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.350889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.352013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.352494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.352990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.353943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:31.354523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.356145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.364210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843e7d4-5658-4b24-8fe8-b41751605c6b None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.364519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843e7d4-5658-4b24-8fe8-b41751605c6b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.364772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843e7d4-5658-4b24-8fe8-b41751605c6b None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.367194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 213/413] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:31.372481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 200/413] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:31.373940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.380107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.380376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.380594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.414008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:31.415381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:31.496771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f89f4-53c6-431a-814b-4903acc2c07d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.536094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.537413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.537920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.538410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] PATH_INFO: `/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.539895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] RBAC: Authorizing `identity:get_group(group_id=fe29b868d0bb4e0ca04d58defb4328e8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.541617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.543986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 214/414] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:31.549896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.569118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 201/415] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:31.597787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.662871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be311-bb8b-462c-8134-68bfddea4cd8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.699033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.700337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.700814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.701326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] PATH_INFO: `/v3/projects/06d48372060e456ea0c20a517b83a4d3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.702771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] RBAC: Authorizing `identity:get_project(project_id=06d48372060e456ea0c20a517b83a4d3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.704411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.710700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 215/416] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/projects/06d48372060e456ea0c20a517b83a4d3 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:31.713097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:31.818129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5634d1f-1ba7-4926-9d03-e8eae04efde7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.822652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb28d-80f5-45b7-b8ed-29626a6d59d3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:31.860136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.861549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.862034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.862592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] PATH_INFO: `/v3/projects/06d48372060e456ea0c20a517b83a4d3/groups/fe29b868d0bb4e0ca04d58defb4328e8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.865431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] RBAC: Authorizing `identity:create_grant(project_id=06d48372060e456ea0c20a517b83a4d3, group_id=fe29b868d0bb4e0ca04d58defb4328e8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:31.867343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:31.882258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:31.883202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:31.883494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:31.883875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:31.891792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 216/417] 10.222.0.83 () {58 vars in 1442 bytes} [Tue May 10 14:49:31 2022] PUT /identity/v3/projects/06d48372060e456ea0c20a517b83a4d3/groups/fe29b868d0bb4e0ca04d58defb4328e8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:31.895492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:31.905047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:31.906024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:31.906858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:31.910530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 202/418] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:31.913000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:32.088635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee3a3f-657c-4313-87cc-66feaadab29b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:32.135310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:32.136379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:32.136857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:32.137348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:32.138326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] RBAC: Adding query filter params (name=member) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:32.138866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:32.140486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:32.151071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 217/419] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:32.156472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:32.274804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2118d-4361-46be-859e-a3980f24bcb8 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:32.321776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:32.322450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:32.322907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:32.323190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:32.395584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:32.397122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:32.501923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 203/420] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:32.503820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:32.634321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17130f3-8095-418d-9b85-9e408f80bf70 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:32.673718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13339100-1732-4674-b352-3962dd08f345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:32.674946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339100-1732-4674-b352-3962dd08f345 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:32.675445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339100-1732-4674-b352-3962dd08f345 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:32.675918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339100-1732-4674-b352-3962dd08f345 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:32.677298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13339100-1732-4674-b352-3962dd08f345 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:32.678944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13339100-1732-4674-b352-3962dd08f345 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:32.681265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 218/421] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:32 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:32.686540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:32.804804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd2b-d6ec-474e-8df6-b54329b4afdb None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:32.845789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:32.846688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:32.846973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:32.847249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:32.848436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:32.849943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13121c-8424-4465-862b-b20fe1e6951f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:32.852523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 204/422] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:32 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:32.858304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13339100-1732-4674-b352-3962dd08f345 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.014373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13339100-1732-4674-b352-3962dd08f345 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:33.059665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:33.060343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.060627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.060902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.061750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=1dbd101bd7884fe899e27996317060c5) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:33.062062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:33.063592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d6b7-b5c2-4e13-bc0f-ef4e2611ca70 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:33.076354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 219/423] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:32 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:33.260183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc196c-257e-47f4-94fe-49d7cf9b8552 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.260183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc196c-257e-47f4-94fe-49d7cf9b8552 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.260183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc196c-257e-47f4-94fe-49d7cf9b8552 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.261014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 205/424] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:33.267086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58840-200f-4fdb-a822-64092588cb6f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.267374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58840-200f-4fdb-a822-64092588cb6f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.267650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58840-200f-4fdb-a822-64092588cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.287901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7da58840-200f-4fdb-a822-64092588cb6f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:33.289081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7da58840-200f-4fdb-a822-64092588cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:33.370880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 220/425] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:33.400566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc196c-257e-47f4-94fe-49d7cf9b8552 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.523781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc196c-257e-47f4-94fe-49d7cf9b8552 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:33.556286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4795-1aa1-4e12-986a-5023978fe19c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.557198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4795-1aa1-4e12-986a-5023978fe19c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.557664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4795-1aa1-4e12-986a-5023978fe19c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.560452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 221/426] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:33.566183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.566498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.566767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.568601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:33.569560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.569836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.570104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.586336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:33.591333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:33.592751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:33.593318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:33.594221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:33.595067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 206/427] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:33.602475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.675867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 222/428] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:33.699380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.731177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85964713-dbec-44fe-bd40-a8962ffd64b3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:33.765824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:33.766520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.766802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.767076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.767811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:33.768458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:33.769476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:33.779027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 207/429] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:49:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:33.785594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.830660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561a410-2840-474c-8907-40a2aa5aef9c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:33.876631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:33.877444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.877726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.878001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:33.888826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:33.894399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0cec2-d2f5-4b5b-a4b9-38206c36d2f0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:33.897066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:33.897966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:33.898829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 223/430] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:33.917766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:33.940962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:33.941950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:33.942502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:33.942989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.022069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.024013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.048294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3acb9-644e-4c91-8ef6-59e9436216e9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.090166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.090873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.091157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.091436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.092354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:34.092666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.094680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.106253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 224/431] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:34.112177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.139357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 208/432] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:34.145021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.219600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275162b9-9467-4b88-a5b1-434e62d3e01f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.243751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25500b2e-4037-4f43-8c24-dd580ef0f733 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.256702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.257232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.257518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.257790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.280627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.281522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.281759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.282026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.283240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.284660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa05d52-e989-42c6-8412-af79aa3c446f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.290595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 209/433] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.290595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775e07a-97b0-4b78-847f-bc4e92f9b7f6 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.290595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775e07a-97b0-4b78-847f-bc4e92f9b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.290595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775e07a-97b0-4b78-847f-bc4e92f9b7f6 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.290595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 210/434] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:34.294090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775e07a-97b0-4b78-847f-bc4e92f9b7f6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.328678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.330045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e1ba2-6195-4c76-b1fb-a0dc33214389 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.395830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775e07a-97b0-4b78-847f-bc4e92f9b7f6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.431099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 225/435] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:34.431099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.431099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.431099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.433118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.433971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.434250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.434559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.435798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.437229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.443538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 211/436] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.443538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.447419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:34.448664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:34.518462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 226/437] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:34.518462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.552357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de4ea-f943-4197-a56f-357cee942aa2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.585629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.586480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.586765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.587081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] PATH_INFO: `/v3/projects/1dbd101bd7884fe899e27996317060c5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.588279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbd101bd7884fe899e27996317060c5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.589739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.591459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 212/438] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/projects/1dbd101bd7884fe899e27996317060c5 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.592218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.624081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360230c-38e7-4f7d-892b-e3660362320d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.657043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.657690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.658027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.658305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.659176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=edf562e307eb48899cd024cddf6d7be3, scope.project.id=4c1520328285426292d433237efc53cc) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:34.659493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.660919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.666519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 227/439] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=4c1520328285426292d433237efc53cc&role.id=edf562e307eb48899cd024cddf6d7be3 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.667288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.695815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6239b-8b5c-438e-a7c7-35aeacd51708 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.731764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.732601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.733176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.733176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.744888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5d1f1198-45d2-419f-93c0-6aeb35acf6de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:34.753313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:34.754319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:34.755188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:34.759626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 213/440] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:34.761411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6583-4c0b-4a5b-a153-e4d6ff4a20fb None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.761700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6583-4c0b-4a5b-a153-e4d6ff4a20fb None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.761974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a6583-4c0b-4a5b-a153-e4d6ff4a20fb None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.767031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 214/441] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:34.767031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a6583-4c0b-4a5b-a153-e4d6ff4a20fb None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.791786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fda57-e609-4eb7-bdbc-b18d83fd56ee None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.833729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.834627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.834914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.835182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.836476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.837891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dac64d-61c7-48ec-a80a-fba4de0fadfa None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.839777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 228/442] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.843787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.844108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.844351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.861478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:34.862693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:34.890009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a6583-4c0b-4a5b-a153-e4d6ff4a20fb None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:34.925200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:34.925865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:34.926150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:34.926455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:34.927220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:34.927510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:34.928879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:34.937333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 215/443] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:34.938334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:34.943010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 229/444] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:34.943834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.033594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a259cf-89d6-4d4f-8c3e-95afca3eb032 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.033814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3944-7b31-4df5-a37d-97dfed73e180 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.065074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.065228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.065714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.065810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.065989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.066077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.066260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.066368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.067100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=1dbd101bd7884fe899e27996317060c5) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:35.067402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.068815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.074296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 216/445] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=1dbd101bd7884fe899e27996317060c5&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:35.075081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.145223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.146759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.178136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc0a1-c5f9-4333-8911-011c2248bfd8 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.214074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.214924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.215203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.215471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.226322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:35.234807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:35.235986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:35.236831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 217/446] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:35.242418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.255635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 230/447] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:35.261322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.344692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067085e-141e-48a0-95a9-d77af07e197b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.357409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fcdc6-f31f-49c5-b942-2b87ae3b9e8f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.379859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.380523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.380813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.381088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.381836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:35.382142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.384659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.391484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.392339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.392615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.392885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] PATH_INFO: `/v3/projects/4c1520328285426292d433237efc53cc` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.393424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 218/448] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:35.398465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] RBAC: Authorizing `identity:get_project(project_id=4c1520328285426292d433237efc53cc)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.398858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.400400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.402358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 231/449] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/projects/4c1520328285426292d433237efc53cc => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:35.407688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.519613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7cd5-e3ec-4e29-b501-d63e530f5ecf None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.523449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49416a9-d540-477e-9577-1a43991bdc38 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.554714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.555559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.555869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.556135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] PATH_INFO: `/v3/groups/d0735f5dc83845e189b80fb5568f715a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.557331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] RBAC: Authorizing `identity:get_group(group_id=d0735f5dc83845e189b80fb5568f715a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.558796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.558925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.559733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.559839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.560105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.565668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 232/450] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/groups/d0735f5dc83845e189b80fb5568f715a => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:35.566202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.635922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.637426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.667507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b6fd-5487-4ff3-834b-266f080320ad None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.698405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.699052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.699328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.699599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.700406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] RBAC: Adding query filter params (group.id=d0735f5dc83845e189b80fb5568f715a, role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4c1520328285426292d433237efc53cc) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:35.700713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.702107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.707724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 233/451] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/role_assignments?group.id=d0735f5dc83845e189b80fb5568f715a&scope.project.id=4c1520328285426292d433237efc53cc&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:35.730719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 219/452] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:49:35.736725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.835668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35613f80-ffbf-4e2e-8b82-19e37293a182 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:35.869889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:35.870800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:35.871080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:35.871354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] PATH_INFO: `/v3/projects/06d48372060e456ea0c20a517b83a4d3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:35.872556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] RBAC: Authorizing `identity:get_project(project_id=06d48372060e456ea0c20a517b83a4d3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:35.873981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:35.875698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 234/453] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/projects/06d48372060e456ea0c20a517b83a4d3 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:35.881412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:35.978563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778292d7-ad83-49b9-b088-e6166d4dab55 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:36.010990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:36.011846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:36.012124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:36.012393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] PATH_INFO: `/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:36.013549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] RBAC: Authorizing `identity:get_group(group_id=fe29b868d0bb4e0ca04d58defb4328e8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:36.015005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce067f46-2626-4d2e-b44d-a9f7ad182b70 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:36.016884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 220/454] 10.222.0.83 () {56 vars in 1180 bytes} [Tue May 10 14:49:35 2022] GET /identity/v3/groups/fe29b868d0bb4e0ca04d58defb4328e8 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:36.022374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:36.116633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da58c-4fd7-4633-a259-437775d6fdac None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:36.151288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:36.151944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:36.152231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:36.152509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:36.153333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] RBAC: Adding query filter params (group.id=fe29b868d0bb4e0ca04d58defb4328e8, role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=06d48372060e456ea0c20a517b83a4d3) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:36.153647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:36.155122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9129274-0ad3-489a-832a-9fda9a2b1b02 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:36.160751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 235/455] 10.222.0.83 () {56 vars in 1376 bytes} [Tue May 10 14:49:36 2022] GET /identity/v3/role_assignments?group.id=fe29b868d0bb4e0ca04d58defb4328e8&scope.project.id=06d48372060e456ea0c20a517b83a4d3&role.id=57e23e25b8c046ba882d07965d852386 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:39.095559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb461e-61a2-4fa5-ab80-ba399992613a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.096077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb461e-61a2-4fa5-ab80-ba399992613a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.096151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb461e-61a2-4fa5-ab80-ba399992613a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.097705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 221/456] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:39.103177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.103590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.103756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.121193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:39.122366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:39.136433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2dc02-300b-4fc6-8bf5-de46f934275e None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.136723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2dc02-300b-4fc6-8bf5-de46f934275e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.137002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2dc02-300b-4fc6-8bf5-de46f934275e None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.138591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 222/457] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:39.143493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a03f-0b28-446c-a39f-434885786c17 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.143781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a03f-0b28-446c-a39f-434885786c17 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.144061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a03f-0b28-446c-a39f-434885786c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.160588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2903a03f-0b28-446c-a39f-434885786c17 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:39.161774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2903a03f-0b28-446c-a39f-434885786c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:39.190248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 236/458] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:39.214571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.228743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 223/459] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:39.230069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddce53-306d-439f-a3d9-318d241cef81 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.230358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddce53-306d-439f-a3d9-318d241cef81 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.234714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddce53-306d-439f-a3d9-318d241cef81 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.235985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 224/460] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:39.241410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.241693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.241972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.272697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:39.273876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:39.312319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d599-949e-4741-b702-0e0e957a775b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.347034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.348396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.348689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.348966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.352328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 225/461] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:39.353204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.359842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.361283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:39.382510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:39.385198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:39.387421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:39.387421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:39.387421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.387421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:39.387421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 237/462] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:39.390268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.472573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e21ad-f4c6-43c8-ba16-88b804ad6979 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.502502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdc75b-626a-454f-a896-422535d2999d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.511116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.511921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.512205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.512486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.522729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.524211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.536208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.537000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.537281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.537552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:39.544716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:39.546715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 226/463] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:39.548810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.549554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.549554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:39.567698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:39.569762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 238/464] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:39.572193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.656063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065c36-2d3c-4fbd-a9c1-41be1ac760bf None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.671105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0ce1d-0124-4ae0-bf04-17b0c4a26d4a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.689036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.689708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.689992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.690265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.691024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:39.691336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.692725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.704277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 227/465] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:39.705092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.710023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.710939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.711418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.711898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.712825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:39.713310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.714915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0db87-c615-4437-be54-51453e29fc44 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.730793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 239/466] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:39.732203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c27e6-0d6c-4029-8ce4-45c948d58731 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.732498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c27e6-0d6c-4029-8ce4-45c948d58731 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.732782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c27e6-0d6c-4029-8ce4-45c948d58731 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.734160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 240/467] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:39.734743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c27e6-0d6c-4029-8ce4-45c948d58731 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.807718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d18fd-8746-4959-98e2-3d07ea14c5c6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.834516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c27e6-0d6c-4029-8ce4-45c948d58731 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:39.840104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.840777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.841064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.841341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.842056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:39.842364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.843770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.859876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 228/468] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:39.859876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:39.873215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:39.873882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.874166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.874473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.875179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:39.876605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d033d03-6897-4b77-86f3-371a70eb7cfd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:39.902606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 241/469] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/users => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:39.905642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:39.906989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:39.907500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:39.937500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:39.939153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:40.000906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d0fd-fcd1-4902-bf87-2911d8ad2677 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.016690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 242/470] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:40.017968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.066153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.067218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.067505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.067782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.068798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.070277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.095482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 229/471] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:39 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:40.096876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.152704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7be689-bb78-4ba1-8e80-fe9af9a95664 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.196836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.197536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.197820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.198098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.198845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.200348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad5ea9-aba5-4850-8d23-f90efc55f8ca None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.216089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e5398-9019-4dc4-8738-95f9dc644a49 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.224571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 243/472] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:40 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:40.226024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f08bd-47f3-4708-86a9-cc87658fa1ab None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.226325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f08bd-47f3-4708-86a9-cc87658fa1ab None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.226617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f08bd-47f3-4708-86a9-cc87658fa1ab None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.227886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 244/473] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:40.233960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.234238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.234548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.255915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:40.257044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:40.262269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.263485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.263994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.264484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.278158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.280146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:40.300229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:40.301173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:40.302050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:40.302050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:40.302050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:40.302050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:40.303910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 230/474] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:40.313346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.340854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 245/475] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:40.363879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.452424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c714-2f51-4107-84ce-7f8276f5194c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.469314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d364733-1b29-4077-810d-ac57f1bec1b5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.491778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.492441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.492718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.493014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.493757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:40.494057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.495471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.507137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.507677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.507959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.508234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.509806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.511206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.511405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 231/476] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:40.518813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.519687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.sql.core [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 14:49:40.519687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 14:49:40.519687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [parameters: {'id': '70e63b6547cb45e584012b1707c51503', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 14:49:40.519687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79936) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application conn.query(q) May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result.read() May 10 14:49:40.536880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return ref.to_dict() May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:49:40.537952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application next(self.gen) May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application yield resource May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application next(self.gen) May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.session.rollback() May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.force_reraise() May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise self.value May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application session.commit() May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 14:49:40.539106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._prepare_impl() May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.session.flush() May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._flush(objects) May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application compat.raise_( May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application flush_context.execute() May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rec.execute(self) May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 14:49:40.540396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:49:40.543108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application conn.query(q) May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result.read() May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:49:40.544098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [parameters: {'id': '70e63b6547cb45e584012b1707c51503', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:40.544884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 14:49:40.545799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.551682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 246/477] 10.222.0.83 () {60 vars in 1131 bytes} [Tue May 10 14:49:40 2022] POST /identity/v3/roles => generated 144 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:40.553437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.645082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2069-0d59-4739-aad2-ddd9f5bb18cf None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.685608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.687103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.687592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.687695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a873-1947-473f-a44e-45e8e0b390d3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.688182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.689311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.691047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.712342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 232/478] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:40 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:40.726009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.726939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.727232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.727548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.739470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-eada75bc-9a4f-4f1d-80da-8203fb135b78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:40.748725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:40.749690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:40.750579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 247/479] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:49:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:40.755826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:40.862815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9eaf0-6587-4c31-b5ef-5861e6bc83c3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:40.898377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:40.899081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:40.899399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:40.899676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:40.900436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:40.900745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:40.902145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40282-6e65-49ec-9f8e-0ecd4cf3803e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:40.913968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 233/480] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:42.746525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddbd1-cff9-42df-9910-0bbc56b7bab3 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:42.747044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddbd1-cff9-42df-9910-0bbc56b7bab3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:42.747153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddbd1-cff9-42df-9910-0bbc56b7bab3 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:42.748703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 248/481] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:42.754800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88832de-aa32-449c-b8e2-3a37c2c346a9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:42.754800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88832de-aa32-449c-b8e2-3a37c2c346a9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:42.754800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88832de-aa32-449c-b8e2-3a37c2c346a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:42.772683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f88832de-aa32-449c-b8e2-3a37c2c346a9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:42.774301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f88832de-aa32-449c-b8e2-3a37c2c346a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:42.844198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 234/482] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:42.867785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cddbd1-cff9-42df-9910-0bbc56b7bab3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.000054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34f373-30a2-4481-bc42-76adfe849e6f None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.000348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34f373-30a2-4481-bc42-76adfe849e6f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.000530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34f373-30a2-4481-bc42-76adfe849e6f None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.002125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 235/483] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:43.007050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d6e5-2d97-4630-bbf4-8c7a6d2300c6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.007340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d6e5-2d97-4630-bbf4-8c7a6d2300c6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.007615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d6e5-2d97-4630-bbf4-8c7a6d2300c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.010836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cddbd1-cff9-42df-9910-0bbc56b7bab3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.036183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-89e6d6e5-2d97-4630-bbf4-8c7a6d2300c6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:43.037359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-89e6d6e5-2d97-4630-bbf4-8c7a6d2300c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:43.068870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.069668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.069949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.070225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.087840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:43.095548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.096429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:43.097229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 249/484] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.105018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.134378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 236/485] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:43.142696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea83b-9ead-45ec-bf5d-dca958d24904 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.143239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea83b-9ead-45ec-bf5d-dca958d24904 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.143716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea83b-9ead-45ec-bf5d-dca958d24904 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.145444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 237/486] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:43.159585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.160128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.160623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.181364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:43.182910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:43.247044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497050-7ce4-4640-88b4-21b9f23cc4d8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.249223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 238/487] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:43.250426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.297759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.298821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.299303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.299795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.300737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:43.301239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.302919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.312072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 250/488] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:43.313169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.382428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafafd4-30e5-4ed6-8a12-df165fa51668 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.415542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243c5b-5553-407a-af07-092461fcf537 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.424643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.425805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.426089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.426366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.439713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:43.449613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.450572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:43.451374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 239/489] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.452056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.459431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.460243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.460525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.460800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.476738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:43.487247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.488226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:43.489030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:43.497799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 251/490] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.497799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.580238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9bd6e-5e22-457f-ab7d-3fe7ca777a25 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.604255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1cb66-6739-477a-8f3e-d3aed35e5bbf None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.616745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.617540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.617823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.618125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.628851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.630400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.639499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.640492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.640977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.641440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.642376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:43.642919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.644468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:43.648576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.649951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.650829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:43.650829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:43.650829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.650829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:43.655006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 240/491] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.656073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.661896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 252/492] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:43.663048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.758982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d24d9-4dc9-4aec-b388-b15a7f68f9bd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.770622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19a79d-160e-44e2-9c66-8573c7bfddd7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.797450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.798138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.798451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.798730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.799471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:43.799776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.801128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.804981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.805666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.805952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.806223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.806958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:43.807263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.808638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368103-c7aa-485c-8a16-f1a9cdc93360 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.813726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 241/493] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:43.813726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.824447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 253/494] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:43.825748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dacc-f70d-4ecb-b0de-ed7c5a8fc6a4 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.826032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dacc-f70d-4ecb-b0de-ed7c5a8fc6a4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.826307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dacc-f70d-4ecb-b0de-ed7c5a8fc6a4 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.830499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 254/495] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:43.830499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16dacc-f70d-4ecb-b0de-ed7c5a8fc6a4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.923432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42616d5e-7d43-41e9-8aed-3c87ef743b0b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.933215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16dacc-f70d-4ecb-b0de-ed7c5a8fc6a4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:43.955311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.956183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.956465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.956741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.965786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:43.966597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.966879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.967139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.967229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:43.968596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.976625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:43.978064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:43.984824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.985780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 242/496] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.991746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e083d16d-a36c-4947-8f61-ca1159720a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:43.995298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:43.996294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:43.997518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:43.997518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:43.997518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:43.997518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:43.997518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 255/497] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:43.997790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aad06-81a9-41a5-a4f3-6fa604a29d0c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:43.997944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aad06-81a9-41a5-a4f3-6fa604a29d0c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:43.998246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aad06-81a9-41a5-a4f3-6fa604a29d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.014205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b42aad06-81a9-41a5-a4f3-6fa604a29d0c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:44.015404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b42aad06-81a9-41a5-a4f3-6fa604a29d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:44.077218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 256/498] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:44.078580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974907a5-0776-4e5b-ba17-b5a51e6949e6 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.078868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974907a5-0776-4e5b-ba17-b5a51e6949e6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.079146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974907a5-0776-4e5b-ba17-b5a51e6949e6 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.080385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 257/499] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:44.081126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974907a5-0776-4e5b-ba17-b5a51e6949e6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.081719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27ccd6-4974-439f-8152-fe9ac05c7427 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.116413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.117196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.117479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.117754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.125596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.127092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:44.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:44.144592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:44.145492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:44.146016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 243/500] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:49:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:44.146984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.181022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974907a5-0776-4e5b-ba17-b5a51e6949e6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.212675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.213332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.213612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.213883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.214632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:44.214941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.216302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed63e7d-28b1-42f0-96e6-ccdad12dcbbf None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.226805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 258/501] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:44.227995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.228286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.228569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.243119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ec27b-75b1-4f48-8025-21d7bd2eba40 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.243871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:44.245033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:44.275567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.276316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.276597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.276871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.277651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:44.277954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.279341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.290235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 244/502] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:44.291036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.310968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 259/503] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:44.311779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.383446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2c08-daf5-4b80-8f64-f0af26e3f446 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.402777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019312e4-e606-499f-8213-c57da7fdc1e0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.414812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.415705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.415984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.416256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.425669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.427137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.434135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37044761-b936-4510-b200-76a885428933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.434827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044761-b936-4510-b200-76a885428933 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.435116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044761-b936-4510-b200-76a885428933 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.435401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044761-b936-4510-b200-76a885428933 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.436242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37044761-b936-4510-b200-76a885428933 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:44.436548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37044761-b936-4510-b200-76a885428933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.437938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37044761-b936-4510-b200-76a885428933 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:44.444268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:44.445256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:44.446206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:44.446206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:44.446206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:44.446206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:44.446206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 245/504] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:44.447016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.473481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 260/505] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:44.474298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37044761-b936-4510-b200-76a885428933 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.544017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e82545-67aa-4d89-9622-5df7dcfef132 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.564733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37044761-b936-4510-b200-76a885428933 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.575547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.576319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.576595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.576863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.583979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.585405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.595923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.596700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.596982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.597250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:44.599173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:44.600232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 246/506] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:44.601344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.609574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.611059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:44.627403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:44.628336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:44.629240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 261/507] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:44.629812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.699104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cc81-f3ca-4e25-8882-e4bdecaeeeb1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.720468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77457c6-b471-4bee-8cb5-84dac386ab16 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.730076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.730895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.731178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.731481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.741400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:44.749698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:44.750869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 247/508] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:44.751874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.756275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.756930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.757207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.757484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.758195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:44.758563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.759929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.770628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 262/509] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:44.771389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.856159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71747e06-bdc2-4aac-bd6f-aa424bec2664 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.863010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c8e45-56bf-4b1c-9b55-0ff03b76b6b6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:44.887516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.888159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.888445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.888719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.893889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:44.894557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:44.894838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:44.895110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:44.895922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:44.896225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.897722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:44.916385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 263/510] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:44.916385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:44.957247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:44.958678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.005921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25360-d99d-4c57-979d-126b881814f6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.036445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.037098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.037381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.037652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.038503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:45.038808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.040724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.049804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 248/511] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:45.050491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.057366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 264/512] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:45.058150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.145011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508cf99-6f64-4490-aa3f-1fe0f4dc9c97 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.148815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f150f5-6931-4322-889b-ce734fa7a3b2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.183169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.183824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.183985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.184133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.184365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.184645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.184931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.185091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:45.185210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.185394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.185936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.186784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.187392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.194282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 249/513] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:45.195083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.207054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 265/514] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:49:45 2022] POST /identity/v3/users => generated 247 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:49:45.208262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.303482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5c630-fbc0-4d6c-b659-10ac2544bc2f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.326303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1c60-5e89-4aa2-a657-28d12fa4bd67 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.337899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.338594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.338872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.339146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.364215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.365028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.365308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.365838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.377607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.379136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:45.396963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:45.397854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:45.398754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:45.398754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:45.398754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.398754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:45.398754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 266/515] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:45.399016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.414935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.416810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.510680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2769a6-966b-4429-b6f0-febe74c3dc26 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.519203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 250/516] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:45.520038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.557411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.558094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.558381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.558689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.630589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.632041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.634930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d57c12-9461-4149-8d62-37101c3b227f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.668009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.668802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.669095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.669367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.678831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.680277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:45.697711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:45.698692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:45.699606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:45.699606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:45.699606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.699606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:45.699606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 251/517] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:45.699880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.732352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 267/518] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:45.733191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.798996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5c1-0ab5-45fd-b8f4-aa018847b4b0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.829064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b115-de5f-42f8-8e63-4f8d715668e8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.831639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.832300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.832581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.832854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.833565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:45.833870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.835259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.850827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 252/519] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:45.850827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.863521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.864775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.865900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.866403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.878277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.879903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:45.898333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:45.899431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:45.900320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:45.900320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:45.900320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:45.900320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:45.900320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 268/520] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:45.900648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:45.957895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53243-1b20-414e-ba90-1b64e994ffde None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:45.992228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:45.993731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:45.994291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:45.994592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:45.995477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:45.995793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:45.997736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.002035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07256c91-f6bb-4b3d-bb5a-171932ba9ac1 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.014153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 253/521] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.014153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.045753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.046583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.046865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.047140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.060462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.062131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:46.082409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:46.083310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:46.084175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:46.084175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:46.084175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:46.084175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:46.084175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 269/522] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:46.084471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.106948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dcc2a-219b-4407-a115-1f019ecbc0d7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.140174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.140971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.141260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.141537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.148006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.149476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:46.163884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:46.165275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.166143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:46.170124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 254/523] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:46.171174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.193299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9db917-23ae-4ee5-ada8-b4e58d0e1626 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.225215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.226078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.226550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.226828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.227535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.227841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.229191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.240148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 270/524] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.244941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.270549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44627475-e6e9-4469-9dae-77a452ecc7b6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.302048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.302870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.303151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.303427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.311214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.312598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:46.326501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:46.327595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:46.328608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:46.331843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 255/525] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:46.331843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.343167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce83f-f6bf-4677-8512-78a8cd4762a7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.381625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.382306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.382621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.383408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.384184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.385704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.386120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.402444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 271/526] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.402444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.442556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d907215-84da-4837-80ca-ed55211f9388 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.476831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.478016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.478300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.478602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.479391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.479701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.481149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.495156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a79-fe01-481b-8bee-39b99eb33a94 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.504478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 256/527] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.504478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.527393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.528179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.528464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.528734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.535086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.536501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:46.550800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:46.551659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:46.552495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 272/528] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:46.552969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.606013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b72ae-4946-4233-b115-9e6d796c4ce0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.638768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.639655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.640068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.640484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.641473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.641923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.643609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.647089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ae65e-e88f-48c6-b686-60f65cdf493d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.679113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.679900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.680187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.680493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.684492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 257/529] 10.222.0.83 () {56 vars in 1139 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.685172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.687130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.688573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:46.702435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:46.703282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 273/530] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:46.704730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.785497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c0a1-f656-4387-9375-2c420299cd44 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.800501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a13ba-e59b-4420-a0a4-b8fb062602b2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.823123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.823731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.824011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.824284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.825110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.825422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.826906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.832089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 258/531] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.832853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.845427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:46.846864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:46.847152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:46.847428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:46.848240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:46.848553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:46.850050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:46.880746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 274/532] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:46.881556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:46.973061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f66a-9b0a-4ee9-a89e-6e7cecb07f35 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:46.989640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a5e1a-e992-459c-bf86-418052e36dab None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.011428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.012463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.012972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.013445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.026308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.027227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.027509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.027808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.028600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:47.028910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.030359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.043408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 275/533] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:47.044149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.089653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.091158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.141688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a58a40-6306-4e2b-bd56-1a4bc4b01004 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.174716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.175391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.175669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.175945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.176761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=f00c2803cc6a4e4a982d05cd419e5301) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:47.177072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.178529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.184793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 276/534] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/role_assignments?user.id=f00c2803cc6a4e4a982d05cd419e5301&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:47.185632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.193889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 259/535] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:47.194893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.289620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016520cb-ec5b-4682-a929-fb9f9ab33439 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.298639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd8ac-eb69-4de0-8f7d-c461e2578932 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.327765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.328449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.328732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.329002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.329845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=e1bcef95380344929f010f445efb1373) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:47.330153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.331661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.338205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.338907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 277/536] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/role_assignments?user.id=e1bcef95380344929f010f445efb1373&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:47.339089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.339312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.339585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.369016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.370559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-0ec3702b-46f6-4c6e-a94a-8a6e47731db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:47.398143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:47.400737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:47.402913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:47.402913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:47.402913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:47.402913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:47.402913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 260/537] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:47.405505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.524015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b411-bfbb-48eb-83ce-965ac9ae9936 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.525025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b411-bfbb-48eb-83ce-965ac9ae9936 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.526100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b411-bfbb-48eb-83ce-965ac9ae9936 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.529407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 261/538] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:47.542757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692620c-bf35-408d-9917-838573c3a11c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.545195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.545483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.545762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.565412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:47.566779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:47.586548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.587860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.588452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.589169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:47.589469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.590876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.608803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 278/539] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:47.619453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.648280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 262/540] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:47.686288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.802956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada073d6-2de5-47be-b63d-c8fb2bc70398 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.846747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.848287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.848734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.849166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.859957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:47.863230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:47.875303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522fe87-e4ed-416a-bc8c-1daec3bb3c74 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:47.879643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:47.880662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:47.881754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:47.884406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 279/541] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:47.894411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:47.971113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:47.972272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:47.972704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:47.973148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:47.995904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efe1ce-db8f-4a16-aecd-bdead7e85662 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.006995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:48.023652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:48.025353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:48.026332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:48.028691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 263/542] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:48.034185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.043646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.044327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.044614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.044895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.045687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:48.045999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.047446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.074270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 280/543] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:48.080689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.144479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b012-502f-448f-8d7a-1f6c9db8683d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.185487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.186083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.186356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.186657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.187405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:48.187711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.189102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.202866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 264/544] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:48.210175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.276241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c500f-7240-4038-a7f0-3cee2c4f9fde None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.335859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.336469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.336783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.337063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.337934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=85f9abef52de40929e0bf2f8e28cd608) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:48.338299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.339752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.347880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 281/545] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/role_assignments?user.id=85f9abef52de40929e0bf2f8e28cd608&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:48.376647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086898cd-4945-4f97-8844-af9f5041fc54 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.411571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.412391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.412672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.412952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.424697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.426264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:48.444381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:48.445273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:48.446408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:48.446408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:48.446408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:48.446408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:48.446408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 265/546] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:48.452633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.581867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add19e21-126d-4ea3-8c28-2d14d3fc1929 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.633200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.633931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.634219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.634523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.635261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:48.635574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.637094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.654193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 282/547] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:48.659784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.787022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da113-8239-4be5-b02a-d42e29f99420 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.820850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.822002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.822531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.822998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:48.830615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:48.832279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:48.846884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:48.847787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:49:48.848666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:48.854571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 266/548] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:48.859457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:48.964373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0f16-8daf-4464-97fa-be48a440c872 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:48.998148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:48.998846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:48.999127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:48.999396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.000226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:49.000529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:49.001938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:49.022498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 283/549] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:49:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:49.025193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:49.158081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc8df-5ee8-483d-8634-bed3dc61ee64 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:49.204484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:49.205146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:49.205434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:49.205713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.292248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:49.294067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:49.407385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 267/550] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:49:49.412814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:49.536976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1062ef5-81dd-4a63-abb3-cd213c240766 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:49.570466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:49.571875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:49.572158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:49.572426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.582564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:49.584010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6791179a-8a41-48b2-ae4d-2752e78aa1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:49.602752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:49.603678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:49.604510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:49.604510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:49.604510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:49.604510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:49.608497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 284/551] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:49.611514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:49.739268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8fbd0-8991-47f0-8950-743c48c0ad93 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:49.739564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8fbd0-8991-47f0-8950-743c48c0ad93 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:49.739727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8fbd0-8991-47f0-8950-743c48c0ad93 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.741472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 285/552] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:49.750808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947e46-490d-4ef1-a802-760f7c472b41 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:49.752138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:49.752428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:49.752703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.783691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:49.785287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:49.812913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:49.813968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:49.814478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:49.814969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:49.815876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:49.816393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:49.817955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:49.835660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 268/553] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:49.843297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:49.924363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 286/554] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:49.949838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:49.998637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dd87d-720f-4702-a983-14175ae2af02 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.038436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.039249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.039536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.040351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.048841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.050303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-617e134d-1644-48c0-a85d-41329b2f1993 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:50.065429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:50.066313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:50.067201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 269/555] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:50.068115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2850d5-d239-4bb5-8f99-cd25168dbbae None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.068115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2850d5-d239-4bb5-8f99-cd25168dbbae None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.068384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2850d5-d239-4bb5-8f99-cd25168dbbae None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.073924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 270/556] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:50.074999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2850d5-d239-4bb5-8f99-cd25168dbbae None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.080583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fec90-f14b-41e8-a229-60dd4dd003a4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.114067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.114901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.115187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.115461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.126507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-06090d89-5a8a-4b7d-8b2d-242b2d63cea6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:50.134428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:50.135315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:50.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 287/557] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:50.136885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add2ee-4d60-40ad-9c75-d29b6c32ff3f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.137453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add2ee-4d60-40ad-9c75-d29b6c32ff3f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.137453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add2ee-4d60-40ad-9c75-d29b6c32ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.153322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-53add2ee-4d60-40ad-9c75-d29b6c32ff3f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:50.154536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-53add2ee-4d60-40ad-9c75-d29b6c32ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:50.180954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2850d5-d239-4bb5-8f99-cd25168dbbae None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.212674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.213358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.213643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.213918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.214749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.215043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.216480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ca9b4-3acf-4d51-a6d1-2a2ba14ed28d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.221200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 288/558] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:50.222722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078931-590a-4116-9d32-976d4f21862e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.223185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078931-590a-4116-9d32-976d4f21862e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.223605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078931-590a-4116-9d32-976d4f21862e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.225773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 289/559] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:50.226644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078931-590a-4116-9d32-976d4f21862e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.239227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 271/560] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:50.240362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.241017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.241412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.263761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:50.264937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:50.338415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 272/561] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:50.338415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.340747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078931-590a-4116-9d32-976d4f21862e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.377262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.377929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.378211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.378512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.379251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.379611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.380965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.388813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 290/562] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:50.388813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.434628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ced3c-a4bc-40cb-9f8a-0f031613538c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.469434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.470099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.470410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.470684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.471504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=2a71e33d66744c208cd0bebb9d09d521) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.471805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.473250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.478825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 273/563] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/role_assignments?user.id=2a71e33d66744c208cd0bebb9d09d521&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:50.479847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.502887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b3478-463c-40ed-b437-3ec1172ef9b7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.540002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.541239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.541792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.542273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.554711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:50.563704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:50.564713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.565516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:50.568474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 291/564] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:50.569562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.597445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247dd2-d546-44f4-b3ad-83652c11ab31 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.634297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.635143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.635673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.635673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.647692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:50.656224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:50.657591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:50.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:50.662352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 274/565] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:50.663338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.686056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20221f-4803-430f-a111-e99788a65e75 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.721175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.721972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.722256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.722559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.733526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.735017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:50.752997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:50.753905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:50.755014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:50.755014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:50.755014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:50.755014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:50.755014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 292/566] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:50.755714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.784950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46ce90-73e6-4af1-ac7b-b9b6a97aaeaa None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.826960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.828781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.829072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.829348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.831093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.831205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.832506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.841315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 275/567] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:50.842153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.863227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1080adb-42cb-4b39-b0ca-03ba30417d5a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.897651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.898701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.899262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.899757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.900701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.901212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.902803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:50.911247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 293/568] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:50.912334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:50.944447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74958-dedf-4a23-bc7a-e1b917ce6a13 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:50.982227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:50.982963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:50.983235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:50.983511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:50.984247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:50.984552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:50.986214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.001662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 276/569] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:51.002218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.026557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b10d8a-f134-4a74-89b9-294d7c15b8f5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.070837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.072730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.073468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.074057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.085382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.087701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:51.105581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.106515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.107399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:51.107399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:51.107399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.107399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:51.109612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 294/570] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.110633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.118924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93884af-42c2-49e5-a96c-84a8b9de4f0e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.159019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.159816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.160104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.160383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.170549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.171984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:51.189105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.190100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 277/571] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.191472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.213582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93d188-fd70-4784-9793-e276664ab15a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.247698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.248559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.248849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.249122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.256525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.257947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-7b896ae7-74ea-4336-b631-63892dc80445 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:51.271940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.273133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:51.274684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 295/572] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.275293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297914d-6c0b-4560-a48f-5469788582f1 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.275576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297914d-6c0b-4560-a48f-5469788582f1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.275846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297914d-6c0b-4560-a48f-5469788582f1 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.281497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 296/573] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:51.282153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297914d-6c0b-4560-a48f-5469788582f1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.298215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6feb5-69a8-4bc4-a129-b873016a7a8f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.332206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.332867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.333149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.333425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.334139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:51.334471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.335902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.348575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 278/574] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:51.349403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.387740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297914d-6c0b-4560-a48f-5469788582f1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.424071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.424721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.424999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.425358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.425969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:51.426271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.427885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046a211-e75e-4138-88e2-d1d0b940a03a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.439957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 297/575] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:51.441089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.441375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.441701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.449375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679bc16-727a-4d05-85bd-563e21e1a713 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.457383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:51.458583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:51.484507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.485173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.485459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.485733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.486588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:51.486900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.488319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.505081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 279/576] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:51.505876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.538546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 298/577] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:51.541523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.608894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3b20-9574-4d26-9896-a73aad6a651b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.650303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c64f9-0433-481f-b406-a47bb16fbfc5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.654932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.655750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.656034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.656310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.663757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.665268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:51.679395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.680349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:51.681281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 280/578] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.682801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.687064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.687874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.688161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.688439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.697004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.698453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:51.712876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.713820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 299/579] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.787984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3ce6-9c61-4342-ada4-2db11ee27103 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.811091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8822f-515e-47b7-b1d5-ee782f3fad14 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.821653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.822461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.822748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.823025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.832555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.833992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:51.843069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.843865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.844147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.844424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:51.850834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.851684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.852560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:51.852560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:51.852560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:51.852560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:51.852560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 281/580] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.852828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.859447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:51.867571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:51.868965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 300/581] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:51.870969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:51.959416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559132-d466-4ae7-b95d-da6b257bce53 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.967659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75dbc-93df-40d3-8cac-eed02a56d17f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:51.994605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:51.994605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:51.994901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:51.995170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:51.996064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:51.996444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:51.998427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.007403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.008080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.008359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.009190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.010020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.010328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.011826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.021143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 282/582] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.021143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.028553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 301/583] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.029179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.113871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1c4b8-781f-4cdc-8de5-f778da13c945 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.124291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e196d82-e62a-4b18-8f8d-e10ba1a76d96 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.149757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.150447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.150720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.151003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.151710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.152018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.153396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.157658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.158321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.158631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.158902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.159638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.159938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.161277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.165319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 283/584] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.166311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.170160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 302/585] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.171223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.265255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed74640-84fb-4a13-9494-a6f3640efac7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.265534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d44-6966-46ca-bf1f-523e4b0b7a66 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.299028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.299824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.300104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.300376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.302496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.302496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.302496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.302496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.313387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.313677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.314900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.315133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:52.332089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.333026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.339093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:52.339093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:52.339093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.339093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:52.339316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.340128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.341002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:52.341002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:52.341002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.341002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:52.341232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 303/586] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.341232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.341367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 284/587] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.341367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.435038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e39-ee27-4fbd-9339-829da58b7d78 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.437137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64bad-038a-444f-941b-f05fc79df078 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.468408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.469202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.469479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.469747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.470336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.471157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.471445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.471729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.476654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.478645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.482838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.484272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:52.493595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.494540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:52.495479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 304/588] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.495991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:52.506623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.507533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 285/589] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.509207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.600337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c04b28-ef3f-4ec3-ba64-570a1f22355f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.610945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22266611-a083-460b-b07d-c42e7d36729c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.634906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.635566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.635848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.636123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.636832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.637135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.638518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.643732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.644397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.644685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.644959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.645666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.645973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.647364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.654804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 305/590] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.654804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.660482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 286/591] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.661157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.753149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4183e-9659-4734-acbe-618b2396fd31 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.760439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bc23f-a8af-4883-b4e7-fb9bac3bd1c5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.784799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.785456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.785741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.786017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.786841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.787152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.788563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.792867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.793586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.793869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.794143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.794884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:52.795185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.796522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.800656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 306/592] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.801426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.811511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 287/593] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:52.811511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.892595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d736a-5721-4980-b1d4-796db9d80a28 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.898324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845bd7c-b5d0-4b53-8edc-8e28ac881ab5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:52.925531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.926321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.926627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.926896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.932995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:52.933222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.933779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.934060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.934333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.934620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.940688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:52.942106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b30fbd45-40ba-4a2b-9552-e97ceb15dd1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:52.948166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.949019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.949959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:52.953917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 307/594] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.955168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386efae-4192-47ba-9779-132adf7408d1 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:52.955454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386efae-4192-47ba-9779-132adf7408d1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:52.955728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386efae-4192-47ba-9779-132adf7408d1 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:52.956236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:52.957223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:52.958419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:52.959612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 308/595] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:52.959612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386efae-4192-47ba-9779-132adf7408d1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:52.961538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 288/596] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:52.961538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.052504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386efae-4192-47ba-9779-132adf7408d1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.053025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d7a4-d9b1-461f-bd23-0ba10a09d553 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.083349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.083783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.084225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.084506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.084598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.084780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.084872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.085080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.092316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.093749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.098703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=5bb3df7bdd8847a1a3e17993b1f466a8, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.100114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905d521-c1dc-42a6-9aec-7c16ebb23f9f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:53.107867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:53.109698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 289/597] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:53.112366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.113956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 309/598] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:52 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:53.120480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.120480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.120480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.134064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:53.135413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:53.210436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 310/599] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:53.210436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.239966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b6c4c-ea06-4bb0-b9f3-98f13bfd8bd0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.275113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.275782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.276057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.276323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.277105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:53.277409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.278883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.292035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 290/600] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:53.292347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.311462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a98344-fe10-4774-9e6a-efe763cb5978 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.345503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.346550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.347086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.347565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.348554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:53.349067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.350733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.362816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 311/601] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:53.363625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.405075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dde1b-9ca5-47bf-858d-6f6ae451a8e1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.437193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.438093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.438582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.439009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.439985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:53.440456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.442033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.459941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a05c4a-a9a2-4d32-97ad-20f8b938976c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.464608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 291/602] 10.222.0.83 () {56 vars in 1139 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:53.464608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.492733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.494289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.494784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.495064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.504737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:53.512617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:53.513735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:53.514790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:53.518615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 312/603] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:53.519428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.566373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5054af-3a7b-46bc-9ba0-0b10f9efef30 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.599966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.601128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.601612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.602088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/f00c2803cc6a4e4a982d05cd419e5301/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.613923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25407c2-0657-416f-b206-5b8c2147b849 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.617651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=f00c2803cc6a4e4a982d05cd419e5301, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.619773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.631747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 292/604] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:53 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/f00c2803cc6a4e4a982d05cd419e5301/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:53.632537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.679248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.681829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.682355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.683268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/e1bcef95380344929f010f445efb1373/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.704358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=e1bcef95380344929f010f445efb1373, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.706170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.735061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 313/605] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:53 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/e1bcef95380344929f010f445efb1373/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:53.736438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.758001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dc02-d586-4da4-a519-c45bcfa800a4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.804982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.806431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.806718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.806990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.819818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.821245 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:53.842473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:53.844070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:53.845914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:53.845914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:53.845914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:53.845914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:53.845914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 293/606] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:53.855789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:53.923435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd858e-39de-4fb6-acfa-b950c5308617 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.961022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:53.961791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:53.962076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:53.962347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:53.963127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:53.963427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:53.964804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:53.970175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f12d-24ba-4cdb-8e7c-0f1bb4acf68b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:53.982517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 314/607] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:54.000101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.012815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.013126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.013413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.013688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.014436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:54.014765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.016200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.030616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 294/608] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:54.036302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.127268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b47a5d-e763-4317-9d99-2080cd8cb66f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.160589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.161395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.161673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.161943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.175866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.177651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.193970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022e11-7187-4325-93a6-0d51a8a26f9d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:54.198410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:54.199346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:54.200275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:54.200275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:54.200275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:54.200275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:54.200275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 315/609] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:54.205869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.279138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.279954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.280234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.280506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.292149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.293622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:54.307409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:54.308395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:54.309311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 295/610] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:54.326415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.338359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412c85-cb8c-4066-a79f-7facdd02d96c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.374887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.375559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.375836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.376108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.376815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:54.377114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.378528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.392240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 316/611] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:54.397542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.500491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23eb72-dd74-4726-a5da-61c31147a188 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.541192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.541998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.542277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.542581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.552579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.554485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.556595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cf43d-f49b-4e6b-8b4d-f972a3260b03 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:54.577730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:54.578777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:49:54.579816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:54.582434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 317/612] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:54.590815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.618497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.619634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.620140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.620636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.621634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:54.622158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.623858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.637385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 296/613] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:49:54.643893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.715721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9353-3ec8-4a43-b3fc-5a374c497741 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.750995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.752085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.752391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.752764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.753597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:54.753901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.755496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.776858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 318/614] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:54.781715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:54.828589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056b39-5287-4b96-a5c1-b53d0963c80f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:54.866504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814c020-7385-404b-8bce-30d002232443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:54.867383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c020-7385-404b-8bce-30d002232443 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:54.868171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c020-7385-404b-8bce-30d002232443 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:54.868449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c020-7385-404b-8bce-30d002232443 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/85f9abef52de40929e0bf2f8e28cd608/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:54.884744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c020-7385-404b-8bce-30d002232443 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=85f9abef52de40929e0bf2f8e28cd608, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:54.886195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c020-7385-404b-8bce-30d002232443 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:54.911044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 297/615] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:54 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/85f9abef52de40929e0bf2f8e28cd608/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:54.941760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb3d86-74da-40af-8e68-ec50a19bc487 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:55.007060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:55.007908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.008191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.009047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.025802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:55.027298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:55.058030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:55.059035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:55.059952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:55.059952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:55.059952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:55.059952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:55.059952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 319/616] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:55.064998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814c020-7385-404b-8bce-30d002232443 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:55.188352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814c020-7385-404b-8bce-30d002232443 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:55.227702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:55.228779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.229267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.229744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.230704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:55.231213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:55.232800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:55.260675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 298/617] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:55.265966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:55.371581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436675-d7c2-43fd-b034-879add91adf0 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:55.411626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:55.412430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.412711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.412987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.420514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:55.421976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:55.440567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:55.441461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:55.442359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 320/618] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:55.447390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:55.584829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e745-d1af-4055-ad01-a3560f9fd68e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:55.635083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:55.636151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.636657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.637134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.638127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:55.638666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:55.640367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6add0df-892d-4c0b-bcdb-ec349b04b5cf None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:55.653551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 299/619] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:55.658944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:55.723648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79104e24-39ba-4095-81c9-e3aab1eca137 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.724269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79104e24-39ba-4095-81c9-e3aab1eca137 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.724760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79104e24-39ba-4095-81c9-e3aab1eca137 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.726638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 300/620] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:49:55.731854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.732378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.732870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.763417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:55.765063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:55.783293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f089c-c47e-49ac-8c12-abb2e8066829 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:55.819930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:55.820871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:55.821152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:55.821423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/2a71e33d66744c208cd0bebb9d09d521/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:55.852029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=2a71e33d66744c208cd0bebb9d09d521, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:55.854033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:55.874177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 321/621] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:49:55 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/2a71e33d66744c208cd0bebb9d09d521/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:55.892193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 301/622] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:49:55.928466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:56.106798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce45cb-293a-4308-bda2-9ba7473d620f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:56.157813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:56.158650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.158935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.159260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.170174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-eb0fff64-377b-47e0-9d2b-b9f6e96c4b3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:56.178043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:56.179020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:56.179870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 322/623] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:56.185186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:56.408422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828d84-8f75-4dda-9750-16de4ed7187c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:56.468101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:56.468872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.469060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.469324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.470071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:56.470378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:56.471775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:56.486288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 302/624] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:56.492347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a424bf-0876-4cf3-beaa-4ad5c56c80fd None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.493325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a424bf-0876-4cf3-beaa-4ad5c56c80fd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.493421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a424bf-0876-4cf3-beaa-4ad5c56c80fd None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.494778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 323/625] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:56.495697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:56.504886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.505256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.505453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.554422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:56.554422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:56.711059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 324/626] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:56.720700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6fe0e-d692-4bc2-abc5-9ef7702772a9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:56.749232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:56.753559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:56.754639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.755117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.755596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.766925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:56.768629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-249b41c0-07a6-4ec7-b4bb-b3ee5bdcc4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:56.786946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:56.787881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:56.788784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:56.788784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:56.788784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:56.788784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:56.788784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 303/627] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:56.791357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08cd9-ad90-4163-b307-15849c73beb5 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.791357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08cd9-ad90-4163-b307-15849c73beb5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.791357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08cd9-ad90-4163-b307-15849c73beb5 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.791827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 304/628] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:56.796731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08cd9-ad90-4163-b307-15849c73beb5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:56.856496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394bf5-d6f9-442c-971d-0957711602fa None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:56.890254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:56.891429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.891905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.892380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.896797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08cd9-ad90-4163-b307-15849c73beb5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:56.903896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d4de35fc-b702-4ef3-9d0d-0b45c4db2f30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:56.912128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:56.913056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:56.913881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:56.916679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 325/629] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:49:56.918090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.918816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.919312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.937064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:56.938555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:56.943797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:56.944463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:56.944745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:56.945021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:56.946260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:56.946605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:56.947964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:56.959426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 305/630] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:56.960230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.008185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 326/631] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:57.009854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.084872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a779fd-128f-471a-86c2-6a3536b6d8a1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.108703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc548f1-0126-4944-8a2e-47ab3b950c01 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.117409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.118611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.118891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.119163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.119999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:57.120197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.121713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.134431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 306/632] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:57.135347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.145845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.146987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.147519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.147998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.155493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.157190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:57.172082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:57.173056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:49:57.174026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:57.176655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 327/633] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:57.177690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.234698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6b33d-d850-42a9-928e-ed1d9e2fa4ae None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.269192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.269912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.270192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.270586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.279997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90702f1-7cbf-4232-9b42-2a6303b97856 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.281024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:57.288865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:57.289807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:57.290831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:57.294760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 307/634] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:57.295485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.319059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.320230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.320762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.321805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.333604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.335458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:57.354945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:57.355903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:57.356803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:57.356803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:57.356803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:57.356803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:57.359098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 328/635] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:57.360096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.403817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0619a-681e-4f43-abd3-265d3cae1e59 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.437565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.438233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.438558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.438836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.439695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:57.439964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.441403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.458444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 308/636] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:57.459015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.461886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2faf3-eb54-4c58-bf2a-7777f4602ce3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.497380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.498901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.499400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.499878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.500811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:57.501321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.502890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.511472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 329/637] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:57.512551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.560874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9c084-c2d9-4510-8dff-09fa6d106555 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.595247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.595934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.596222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.596497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.597237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:57.597658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.599003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.615355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f8cd-8e29-4a72-bf1b-209b73e33f47 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.616601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 309/638] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:57.617549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.651048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.652095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.652620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.653119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.720545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.721374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e4c0-748d-417a-b4ea-4765f4326cd1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.722590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646656-bba0-40fa-8aa4-8f0e4aeed895 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.756507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.758087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.758432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.758708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.772006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.774410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:57.794233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:57.795373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:57.796473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:57.796473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:57.796473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:57.796473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:57.797991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 310/639] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:57.798943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.830981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 330/640] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:57.832894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b877c6-fd95-4bd2-aaad-e132e73c5e2f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.833261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b877c6-fd95-4bd2-aaad-e132e73c5e2f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.833604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b877c6-fd95-4bd2-aaad-e132e73c5e2f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.835394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 331/641] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:57.837421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b877c6-fd95-4bd2-aaad-e132e73c5e2f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.899544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2eb2-67ea-4a9d-b1fa-0038756645c0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.939831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b877c6-fd95-4bd2-aaad-e132e73c5e2f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:57.939997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.940642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.940924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.941197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.949297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.950753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:57.965360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:57.966568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 311/642] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:57.967779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:57.976706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:57.978070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.978352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.978651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:57.979446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:57.979752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:57.981239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8920c28-998c-4246-86a8-3594e29befd6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:57.997266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 332/643] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:57.997266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:57.997266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:57.997266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.010751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:58.011910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:58.073152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4dfce-2f30-40ca-bfa8-76086235ea1b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.087097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 333/644] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:58.087097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.126078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.127049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.127433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.127782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.138256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.139849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:58.157207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:58.158486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:58.159748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:58.159748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:58.159748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:58.159748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:58.160701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 312/645] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:58.161334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.194274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eb267-5b66-4e72-a2bf-dfe47619d01c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.228348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.229011 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.229288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.229557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.230380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:58.230793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.232220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.250784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 334/646] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:58.255721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.261039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4722b-356e-4866-b8ff-65345eccc8f6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.293203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.293988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.294266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.294566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] PATH_INFO: `/v3/users/glance` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.301915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.303369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:58.318090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:58.319418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:58.320409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 313/647] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:58.321916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.357058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34627002-c95d-4d11-92d4-139170d0895a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.388933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.389715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.389996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.390264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.400320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:58.408018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:58.409125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:58.410003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 335/648] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:58.410771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.422522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b289-5cb3-4aff-9114-a38e5a53bddc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.454016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.454710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.454994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.455270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.455980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:58.456287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.457654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.469416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 314/649] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:58.470194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.503453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411454c4-dde7-425e-bd8d-2991b0ff017c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.534883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.535527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.535810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.536086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.561904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32900b6-23a0-4c25-ab24-d253a48e0551 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.593431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.594071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.594348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.594647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.595457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:58.595758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.597170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa99746-ed1d-4cb6-b4a1-0414b7224bde None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.598288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.599745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.613262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 315/650] 10.222.0.83 () {56 vars in 1137 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:58.614588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af2d9-b3f4-410a-a968-4f6f476a8d6f None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.614986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af2d9-b3f4-410a-a968-4f6f476a8d6f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.615267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af2d9-b3f4-410a-a968-4f6f476a8d6f None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.616753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 316/651] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:49:58.617445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af2d9-b3f4-410a-a968-4f6f476a8d6f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.691284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 336/652] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:58.692394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.713068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af2d9-b3f4-410a-a968-4f6f476a8d6f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.744741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.745391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.745673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.745946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.746701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:58.747008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.748362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.755833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 317/653] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:58.756593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.790138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a87326-bd99-4e26-9cdb-4b5bc1a950b4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.822419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.823219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.823502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.823775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.830280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.831790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8a6ca308-d4c9-436b-8fdc-0fe084eb335b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:58.845241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:58.846305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 337/654] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.852188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.852969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.853057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449b4fa-b68f-4a85-ac28-9b9c875fe2e7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:58.864368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:58.865485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:58.881864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:58.882533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:58.882816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:58.883093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:58.929226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 338/655] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:49:58.929965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:58.947855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:58.949445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.021149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c65604-988b-483d-8c66-b8c1145a6ef7 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.040617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 318/656] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:49:59.041281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.052468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.053263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.053546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.053822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.064847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.066275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:59.083041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.084100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 339/657] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.085026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.133648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfad3c-5a27-406a-bb2a-a1a7c9fc3785 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.165005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.165882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.166226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.166717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.176009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.177452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.179103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645331d-cdfa-4276-8bba-fd726028dac3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:59.194642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.195721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.196818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:59.196818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:59.196818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.196818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:59.199635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 319/658] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.199635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.211426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.212080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.212364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.212642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.213429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.213737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.215182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.226033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 340/659] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.226830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.291954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc550e-526b-47ca-b243-1c26056ed42b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.318350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfbb23-f2ec-4dcd-acb4-a9caada4135a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.323242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.324024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.324311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.324576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.334162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:49:59.343376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.344671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:59.345673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 320/660] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.346565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.357352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.358154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.358465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.358745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.369359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.370904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:49:59.387233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.388567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 341/661] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.389704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.441120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1e16-0b01-48ca-9b68-2a875a3ee29b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.472658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.473311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.473589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.473870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.474616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.474916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.476287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.481531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f7db8-43c3-4a30-b1ce-27358161fc3c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.488187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 321/662] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.488991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.517591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.518334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.518649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.518927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.519638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.519946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.521314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.533711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 342/663] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.534512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.579280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2cc7-a825-4585-abc5-b36dfac721fe None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.610772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.611403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.611677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.611946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.612754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.613060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.614490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.620189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 322/664] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.620939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.625378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f65a11-8e65-450c-a51f-00c288e1d1d6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.660833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.661429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.661706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.661973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.662770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.663070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.664497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.673435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 343/665] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.674272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.739596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bf817-0049-4db8-b944-211641a0460b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.773882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.775133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.775416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.775690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.776413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:59.776748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.778129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.780549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771edf-41ee-4feb-aa91-e8d8e0188284 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.790370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 323/666] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:59.791588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.825041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.825961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.826137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.826853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.834861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.836330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:49:59.852063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.853033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:49:59.853949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:49:59.858014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 344/667] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.858674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.897006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98de8b2-2d7b-4719-bd6b-4bdd87e0038d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.930887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:59.931688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:59.931971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:59.932249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:59.939288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:59.940738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:49:59.955578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:49:59.956550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:49:59.957486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 324/668] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:49:59.958201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:59.966992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef2fa6-1388-4017-94c1-e67b542cf487 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:59.999320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.000099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.000380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.000651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.010792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.012335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:00.030101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:00.031220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:00.032121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:00.032121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:00.032121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:00.032121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:00.034698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 345/669] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:00.035360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.055955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4902bb-bd9b-4ce8-8663-51eec83d7b26 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.088542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.089475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.089862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.090232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.096925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.098488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:00.112393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:00.113469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:00.114591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:00.120881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 325/670] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:00.120881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.136102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f52b-12b6-4fed-868e-f21bb77cee4a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.168263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.168965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.169311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.169651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.170542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.170928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.172417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.184342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 346/671] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.186507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.213818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7337d-60be-4749-acb1-69bb39c09e54 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.246622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.247288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.247567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.247840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.248664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.248976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.250419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.271585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 326/672] 10.222.0.83 () {56 vars in 1139 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.271585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.286063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6944fb-0fbc-4130-9974-1d04e951abfd None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.318154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.318918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.319274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.319600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.320362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.320727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.322681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.338515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 347/673] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.338515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.363945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d351d7-ae7f-4771-8cdf-1babd107f042 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.398074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.398782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.399069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.399347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.400114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.400420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.401826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.418373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 327/674] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.419003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.432043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4440-55fd-4fd8-ae81-e60c4ccb080a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.471207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.471954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.472243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.472518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.473355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=f00c2803cc6a4e4a982d05cd419e5301) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.473652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.475101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.490416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 348/675] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/role_assignments?user.id=f00c2803cc6a4e4a982d05cd419e5301&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.490416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.543695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd01b35-b47d-4b4b-82fe-dc0f592f86d5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.585963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a22ad3-f124-460a-b93a-93b5f3b20278 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.587276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.588201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.588582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.589810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.628392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.629300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.629651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.629995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.638271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.639884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:00.655857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:00.657106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:00.658266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:00.659561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 349/676] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:00.664260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.672390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.673837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.778364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032dfe0-ffef-4d74-8a64-154276238574 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.785980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 328/677] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:50:00.786705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.824514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.825181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.825462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.825735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.826600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=e1bcef95380344929f010f445efb1373) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.826912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.829127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.834549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 350/678] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/role_assignments?user.id=e1bcef95380344929f010f445efb1373&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.835334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:00.907084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad493f-4693-4044-b641-c2988f8e58f2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:00.948494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:00.949806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:00.950333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:00.950851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:00.951925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:00.952465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:00.954186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:00.976366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 329/679] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:00.983652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.072472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64487-10a4-48ad-b9c8-f91c17331a9e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.110824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebb9a-537f-41e0-aba5-1204a8425a47 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.147156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.148012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.148302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.148580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.152162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.152916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.153266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.153610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.165682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.168685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:01.194823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:01.196280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:01.197201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:01.197201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:01.197201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:01.197201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:01.199844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 351/680] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:01.205659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.241843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.244322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.311282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc5ec-e02c-410c-b7eb-a02f08a336f5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.358278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.358962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.360244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.360244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.360244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:01.360476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.361822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.365453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 330/681] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:50:01.371303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.374327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 352/682] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:01.379926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.471955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761382-3207-40db-a0ea-702311b78898 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.482020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddcc15-e171-43f2-ae56-e3d9c6973844 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.506702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.507483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.508818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.508818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.520978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.522960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.524522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.525489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.525910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.526344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.533402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.535076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:01.542082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:01.543096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:01.544186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:01.544186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:01.544186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:01.544186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:01.545809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 331/683] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:01.551338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:01.552544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:01.553382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:01.554253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:01.556369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 353/684] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:01.563413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.662079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37019-c304-4f2e-9449-27d566e0f730 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.677099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65cf47-f93c-47a5-b731-5faa5cae1ac6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.727688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.728728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.729179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.729666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.730719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:01.731039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.732942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.749075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 354/685] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:01.756082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.843791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.844761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.845364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.845782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.846697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:01.847151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.848734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.862038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e8db0-c19a-4004-abda-4563f574c13b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:01.868217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 332/686] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:01.876394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:01.959684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:01.960360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:01.960644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:01.960915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:01.961793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=85f9abef52de40929e0bf2f8e28cd608) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:01.962104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:01.963578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:01.980044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 355/687] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/role_assignments?user.id=85f9abef52de40929e0bf2f8e28cd608&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:02.006827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f2207-a9cd-4d68-9b3a-fd94bd4c1dd2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.062085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.062934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.063212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.063481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.071206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:02.072674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-70e0fb7e-06e1-4dd1-ac86-25783252e7ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:02.093339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:02.094445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:02.095344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:02.099049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 333/688] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:02.104995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:02.204597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68c567-5a7b-4cd0-ab56-9bf7f33091d0 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.204914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68c567-5a7b-4cd0-ab56-9bf7f33091d0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.205135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68c567-5a7b-4cd0-ab56-9bf7f33091d0 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.206500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 334/689] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:02.211512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.211830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.212064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.229066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3c328-ba5d-4893-838e-4e06cb1cde5f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.231067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:02.232200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:02.268585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.269298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.269581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.269854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.270681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:02.270988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:02.272403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:02.286487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 356/690] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:02.294691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:02.299680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 335/691] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:50:02.327646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:02.423273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7936-f6ae-4e52-b4bc-51caa1caa462 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.430817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a0343-446f-48aa-aa6f-47e806ff9af2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.465356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.465884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.466593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.466710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.466970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.467246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.467423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.467837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.468994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=2a71e33d66744c208cd0bebb9d09d521) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:02.469472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:02.471336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:02.478549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:02.479660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 357/692] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/role_assignments?user.id=2a71e33d66744c208cd0bebb9d09d521&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:02.486483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:02.487454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:02.488301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 336/693] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:02.495706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:02.649579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135667-f8e0-487c-b480-322b830e6cfc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.700350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.701764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.702195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.702644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.703545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:02.703999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:02.705530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:02.715452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 358/694] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:02.722431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:02.860063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b7642-caf1-4169-9b39-0ee20fe00d2a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:02.899885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:02.900695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:02.900974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:02.901246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:02.925411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:02.926923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:02.946485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:02.947439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:02.948343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:02.948343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:02.948343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:02.948343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:02.948343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 337/695] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:02.953199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.063587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f9866-f36b-41c3-a173-c73e6925d431 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.099090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.099757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.100038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.100311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.101027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:03.101332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.102744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fb054-870b-4f18-ab41-59866641e5c6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.115129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 359/696] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:03.121232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.125869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d79-2abc-4ca3-9102-473847881626 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.127115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d79-2abc-4ca3-9102-473847881626 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.127408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d79-2abc-4ca3-9102-473847881626 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.129029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 360/697] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:03.134547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.134832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.135112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.156553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:03.157900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:03.230452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 361/698] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 14:50:03.264542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.319673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863201f-928d-4a6a-88e9-d3c4f803d75e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.370008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.370755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.371031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.371298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.390310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.391761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:03.422333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:03.423360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:03.424393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:03.427472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 338/699] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:03.429333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.458304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b1f1f-0aa5-4816-8943-b0483df0dec8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.551886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.552711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.552989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.553263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.553919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.555859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.560428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b0d46-c9aa-4eef-8541-0a6755178a8c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:03.590294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:03.592622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:03.594685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 362/700] 10.222.0.83 () {56 vars in 1129 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:03.602550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.603235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.603440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.603606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.603762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.604583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:03.604883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.606299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.627759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 339/701] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:03.633330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.719225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8742ed-64a2-41e8-a3a5-b6ae242bac22 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.756158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.756878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.758421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.758421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.758421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:03.758421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.766718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.774694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 363/702] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:03.781115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:03.820137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa8931-3920-4c33-aa97-fdbacdcd8fef None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.879706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.880847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.881359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.881844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.926424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9ec19-852d-4847-8cf5-0842a16eb6cc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:03.954942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:03.955692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:03.955904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:03.956164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:03.956882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:03.957186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:03.958619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f7ecd-f92e-48dd-905c-8ed41e66d902 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:03.966368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 364/703] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:04.018348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.019862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.163167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 340/704] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 14:50:04.164241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603194a-7f9c-45dd-8e19-3162435df8dc None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.165099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603194a-7f9c-45dd-8e19-3162435df8dc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.165378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603194a-7f9c-45dd-8e19-3162435df8dc None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.167058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 365/705] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:04.168985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.172127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.172484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.172831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.193509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:04.195418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:04.279824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 366/706] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:04.297986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55f6c-6f53-4f18-acd2-0b2598bfe95a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.325458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.339047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.339623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.339887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.340156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.353018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.354490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:04.372478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:04.373455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:04.374382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:04.374382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:04.374382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:04.374382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:04.374382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 341/707] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:04.379397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.473730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396263-e70e-4436-ac3c-98d7d93dfb44 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.507686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.509348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.509723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bf578-5a95-49c4-a1b5-1aabedaa4d6d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.510077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.510600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.520833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.522565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:04.540792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:04.541738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:04.542660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:04.542660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:04.542660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:04.542660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:04.544887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 367/708] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:04.555899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.580338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.581304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.581304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.581574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.582297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:04.582751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.584325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.611068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 342/709] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:04.616638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.699266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0149064-e6e2-4cdb-96cd-86bce027768c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.761692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.763172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.763658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.764128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.765064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:04.765568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.767217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42658ed2-5882-4e3f-a135-9c916a80e9bc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.775120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0c0a-3dfc-4eb0-a085-e8084b3b4b24 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.784541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 368/710] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:04.786087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41470-205d-4d68-9637-15bbd1766ed1 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.786600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41470-205d-4d68-9637-15bbd1766ed1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.787043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41470-205d-4d68-9637-15bbd1766ed1 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.792498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 369/711] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:04.793381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41470-205d-4d68-9637-15bbd1766ed1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.814186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.815158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.816026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.816374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.825763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.832184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6192c5e6-05fd-47d8-855c-1a837c89688b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:04.849653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:04.850806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:04.851720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 343/712] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:04.853526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.853884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.854228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.875740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:04.877441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:04.903688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41470-205d-4d68-9637-15bbd1766ed1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:04.951501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:04.952975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:04.953761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:04.954241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:04.955670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:04.957925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:04.966651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 344/713] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:04.967693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:04.983617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 370/714] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:50:04 2022] POST /identity/v3/users => generated 254 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:04.995980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:05.146429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0f32-b19b-4ed9-8fe2-f20f886f1fe3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.182511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.183318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.183602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.183876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] PATH_INFO: `/v3/services/network` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.184832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:05.185883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:05.197762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e54610-691e-40c6-84b2-d39bfd68f5dc None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:05.198036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:05.199028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:05.199888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 371/715] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:05.205541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:05.243864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.244527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.244802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.245075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.245856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:05.246159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:05.247761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:05.260531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 345/716] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:05.266262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:05.311070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647dce-11ff-43e9-be53-20a54d170fdd None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.354544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.355220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.355497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.356243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.356961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] RBAC: Adding query filter params (name=network) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:05.357264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:05.359332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:05.365331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 372/717] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:05.371112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:05.388338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1034-9c36-452d-8d2d-beefbce697ad None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.424810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.425445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.425735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.426008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.426949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:05.427265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:05.428814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4774e1-df52-45a0-930c-fcd4a9a5462e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:05.435363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 346/718] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:05 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:05.496274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c1708-5fe6-4b02-8ce1-e9e333182637 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.532273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.532939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.533217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.533488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.534187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] RBAC: Adding query filter params (type=network) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:05.534515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:05.535927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf250efe-9097-481c-a535-05ee1575441a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:05.541418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 373/719] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:05.640771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107dc0-abca-4233-b676-7178c96dba09 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.641293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107dc0-abca-4233-b676-7178c96dba09 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.641738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107dc0-abca-4233-b676-7178c96dba09 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.643385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 347/720] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:05.650639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.650932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.651194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.670314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:05.671555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:05.754127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 374/721] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:05.789404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107dc0-abca-4233-b676-7178c96dba09 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:05.924857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107dc0-abca-4233-b676-7178c96dba09 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:05.970426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:05.971645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:05.972136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:05.972632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:05.983428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:05.991672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:05.992631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:05.993591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:05.996286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 348/722] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:06.001632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:06.150663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afff60-17c0-4475-9676-9ed6c61cae42 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:06.193121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:06.193803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.194085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.194357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.195181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:06.195515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:06.197404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:06.210080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 375/723] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:06.215265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:06.356208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9a9b-0b7e-403c-952f-b1cf133a2163 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:06.395111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:06.396540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.397059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.397539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.413547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:06.415867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:06.440040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:06.440973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:06.441890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:06.441890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:06.441890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:06.441890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:06.443797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 349/724] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:06.449376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:06.587045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64efd0b-8a7d-40e2-a80d-e8d42dc03b22 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:06.635616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:06.636288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.636568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.636841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.637554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:06.637856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:06.639236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5238f5-5bab-4939-b862-5cc617f65cb7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:06.657919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 376/725] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:06.663450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:06.783920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eb191-3efc-4b3f-86e3-128486fb0489 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.784217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eb191-3efc-4b3f-86e3-128486fb0489 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.784384 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eb191-3efc-4b3f-86e3-128486fb0489 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.786022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 377/726] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:06.787143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7dd55-c42b-4594-9c15-49cbf7e0d254 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:06.791525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.792057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.792590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.822128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:06.823320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:06.828192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:06.828990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:06.829269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:06.829546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:06.842953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:06.844393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:06.865243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:06.866548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:06.867563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:06.868108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 350/727] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:06.877734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:06.960008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 378/728] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:06.995393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:07.046230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98954919-f8cd-45b1-aa55-50db056bc145 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:07.108181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:07.109275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.110461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.111059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.113138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:07.114304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:07.116924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:07.145512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cc011-b84c-4596-b4a2-e702c21e8285 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:07.149161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 351/729] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:50:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:07.154310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:07.212881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:07.213906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.214410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.214869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.215797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:07.217325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:07.246851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 379/730] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:06 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:07.295983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd60d8-0ef5-46da-acae-d1376d565c4d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:07.347319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:07.348364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.348830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.349296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.450242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:07.451748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:07.572982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 352/731] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:07 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 14:50:07.578449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:07.686261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce8aeb-d1e4-4300-941f-0009cbf3a014 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:07.722740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:07.723552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.723830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.724152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.736197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:07.737631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3810142a-d45a-4f22-8258-a05aea42c8c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:07.754942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:07.755985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:07.757543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:07.757543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:07.757543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:07.757543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:07.757543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 380/732] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:07.762075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:07.854192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a716-a9f0-4e5d-8ad7-fa1725271ff3 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.854636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a716-a9f0-4e5d-8ad7-fa1725271ff3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.854800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a716-a9f0-4e5d-8ad7-fa1725271ff3 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.856248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 381/733] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:07.861612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.861904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.862183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.880509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921b03-f7fd-4381-8dcd-bf44afd7b416 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:07.882036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:07.883438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:07.952102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:07.952693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:07.952975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:07.953247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:07.959179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:07.959393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:07.959600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 382/734] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:07.960752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:07.974262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 353/735] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:07.979585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:07.987183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.091172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e64c18-b8ac-47b7-ae4e-778040551242 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.129991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.131137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.131621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.132085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.153469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:08.161738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:08.162723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:08.163567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:08.165981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 354/736] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:08.171611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.196613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe056c6a-2769-49c7-84dd-e8f5cd2ab1bb None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.244101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.244843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.245129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.245402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.253582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.255133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-29c87be1-f68d-4b5d-ac24-4439eed10250 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:08.275302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:08.276307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:08.277222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 383/737] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:08.278447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e72a-884c-45cd-9bd5-db6900782c72 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.278740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e72a-884c-45cd-9bd5-db6900782c72 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.279015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e72a-884c-45cd-9bd5-db6900782c72 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.285286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 384/738] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:08.286210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e72a-884c-45cd-9bd5-db6900782c72 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.310741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d645-b3c0-473b-ad16-a176c93c224a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.353773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.354752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.355040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.355318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.356984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:08.357301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.358706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5756e2-3d5b-4905-b710-31185c58d8e1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.370062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 355/739] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:08.371578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c970-fada-4a87-9196-be7031d87b14 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.371871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c970-fada-4a87-9196-be7031d87b14 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.372150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c970-fada-4a87-9196-be7031d87b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.389152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e560c970-fada-4a87-9196-be7031d87b14 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:08.390399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e560c970-fada-4a87-9196-be7031d87b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:08.397328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e72a-884c-45cd-9bd5-db6900782c72 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.440556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.441226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.441508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.441781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.443051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:08.443366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.445283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.460139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 385/740] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:08.461010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.461510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 356/741] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:08.463014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e4f17-7eae-4b2a-babe-7b2c490dc1b2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.463304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e4f17-7eae-4b2a-babe-7b2c490dc1b2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.463583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e4f17-7eae-4b2a-babe-7b2c490dc1b2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.465070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 357/742] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:08.468648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e4f17-7eae-4b2a-babe-7b2c490dc1b2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.561832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f6563-da3e-4198-a946-13e2b600bff9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.573897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e4f17-7eae-4b2a-babe-7b2c490dc1b2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.595887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.596936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.597218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.597493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.606786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.607028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.607435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.607713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.607982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.608451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.608793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=2a71e33d66744c208cd0bebb9d09d521) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:08.609097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.610523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024bf21-6453-4235-837b-cd2490d94670 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.626718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 358/743] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/role_assignments?user.id=2a71e33d66744c208cd0bebb9d09d521&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:08.626718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.626718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.626718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.635381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:08.636596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:08.639312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:08.640218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:08.641113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:08.641113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:08.641113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:08.641113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:08.646577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 386/744] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:08.649541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.718834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 359/745] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 14:50:08.719818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.781169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314bb55-03c5-41a3-9b29-929ecf7e9bb5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.825214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.826842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.827911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.828396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.841913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:08.852206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090284-756e-4f4e-8676-8fd81a6e5f3b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:08.853615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:08.854587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:08.855430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:08.856065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 387/746] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:08.861486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.902212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:08.902914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:08.903195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:08.903467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:08.904194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:08.904494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:08.905857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:08.925607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 360/747] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:08.925607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:08.984576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9ba8-b468-4764-9103-574ccd817ccc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.018606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.019331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.019618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.019894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.020671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.022054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.028346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09eac-bf9c-409c-af49-956fa98928f3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.042617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 388/748] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:08 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:09.043608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.091867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.093283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.093828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.094317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.095333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:09.095898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.097515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.113973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 361/749] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:09.120145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.152347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f3d45-ed5f-41f6-8443-730902a859da None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.187082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.187880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.188161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.188432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.197251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.198755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:09.215168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:09.216202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:09.217166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 389/750] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:09.222438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.238899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d955e-da7d-4ea9-a94b-12b4d58c1e76 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.275555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.276753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.277255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.277728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.290333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.292227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:09.310238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:09.311259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:09.312208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:09.312208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:09.312208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:09.312208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:09.318509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 362/751] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:09.320359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.328808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02be5f9-1adc-4c67-9d06-56a0965ab572 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.363611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.364244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.364530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.364805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.366176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:09.366289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.367766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.386121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 390/752] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:09.391411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.424547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f732768-3c25-44d5-97a4-db749abedea9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.459225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.460236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.460772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.461825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.462784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:09.463294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.464850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.478112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 363/753] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:09.484001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.495719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121484c-aff2-42ca-8907-84e2d45ee9af None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.542473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.543125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.543409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.543676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.625386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eebd7-33d1-4c23-bdff-9b2d13ece71e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.629433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.631249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.671102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.671904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.672187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.672461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.679901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.681344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:09.713532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:09.714592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:09.715639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 364/754] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:09.720328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.735836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 391/755] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 14:50:09.743394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:09.843738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e6717-78ed-4dfd-abeb-80bbd23a0bbc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.911076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:09.911680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:09.911963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:09.912237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:09.913079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:09.913389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:09.918410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:09.940707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f2cb0-75db-4d97-b888-e30e2ed1c860 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:09.958192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 365/756] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:09.963941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.007789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.009203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.009484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.009809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.034088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.035634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:10.074963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:10.075936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:10.076830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:10.076830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:10.076830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:10.076830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:10.076830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 392/757] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:10.082244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.144879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd666b-dfe6-4d5d-8b9d-3a3eb7d028fb None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.178267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.179072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.179348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.179614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.190432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.191912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.209042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28aa65-e0ad-43de-835e-f199f27dd242 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-fe351c8c-4825-4cb4-94f7-f68e647d331c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:10.209650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:10.211656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 366/758] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435ec2-8d42-44f0-885a-c0d18ee02636 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.213762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435ec2-8d42-44f0-885a-c0d18ee02636 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.218451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435ec2-8d42-44f0-885a-c0d18ee02636 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.222735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 367/759] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:10.222735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78435ec2-8d42-44f0-885a-c0d18ee02636 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.332926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78435ec2-8d42-44f0-885a-c0d18ee02636 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.370807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.371465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.371745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.372110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.372907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:10.373213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.373530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.374260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.374634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.374798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a4e43-8386-4ca6-88d2-a2008880bb02 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.375049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.375984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:10.376365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.378235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.390419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 393/760] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:10.391201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 368/761] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:10.391736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.392026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.392177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.392337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.416416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:10.418137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:10.503040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381826-5f7e-4379-a062-5eb2c3b0334f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.506619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 394/762] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:10.507594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.535838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.536624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.536901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.537173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.545741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.554439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:10.567556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:10.568467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:10.569450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 369/763] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:10.570127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.630167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5f78f-1a4e-4afa-adb7-9f6e9ebd352d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.674913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.676015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.676479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.676922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.678436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001118fa-cafa-48f2-8fd7-0a0511d0b81f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.684344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.685985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:10.699667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:10.700599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:10.701551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:10.708448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 395/764] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:10.709456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.716197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.717090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.717444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.717790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.718547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.719988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:10.739012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:10.739962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:10.741081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:10.743605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 370/765] 10.222.0.83 () {56 vars in 1129 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:10.744592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.829214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d774678-c980-461c-9f07-f571a4247412 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.844772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586316f2-3e5d-44b8-92da-44f66ecbcbce None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:10.865458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.866693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.867008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.867427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.868307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:10.868715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.870228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.880080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:10.880962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:10.881388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:10.881804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:10.882770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:10.883263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:10.888802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 396/766] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:10.888802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.889784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:10.908729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 371/767] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:10.909909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:10.983493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb472b-b993-4609-8b46-88d8a993b828 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.017237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee07309-b0c8-4069-a336-fb9233c6cbb7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.022729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.023549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.024403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.024745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.025520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:11.025891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.027708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.032482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 397/768] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:11.033455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.054692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.055881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.056385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.056854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.057891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=7fc10eb4012647b9874edca4482f7c14) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:11.058414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.060018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.074694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 372/769] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:10 2022] GET /identity/v3/role_assignments?user.id=7fc10eb4012647b9874edca4482f7c14&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:11.076027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.141790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763f50-bda9-4710-a6a1-e9c4c1f8903c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.181595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.182756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.183052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.183328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.185399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=5bb3df7bdd8847a1a3e17993b1f466a8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.186881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd5c0d-c242-41c7-84e3-3de46d035c0a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.203075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 398/770] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:11 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:11.204759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a8fd-9b55-41ac-8c52-e4892ccd9cff None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.205296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a8fd-9b55-41ac-8c52-e4892ccd9cff None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.205657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a8fd-9b55-41ac-8c52-e4892ccd9cff None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.211977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 399/771] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:11.213638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.214007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.214360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.228901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629ad6-b89a-4539-bc30-2c1b96999aa8 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.232496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:11.233793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:11.290112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.290867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.291150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.291426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.294352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:11.294821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.298815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.311592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 373/772] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:11.317084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.353426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 400/773] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:11.409962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.484424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b4fd-94fd-43b0-bf68-a5ba329a11a2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.528951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.529625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.529903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.530175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.531153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] RBAC: Adding query filter params (interface=public, service_id=6d2f25000ae14d3ca796a0b30028088d, region_id=RegionOne) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:11.531463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.533142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.544000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 374/774] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/endpoints?service_id=6d2f25000ae14d3ca796a0b30028088d&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:11.549792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.550890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aabad-fb2a-46b6-96d3-de165b197454 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.585344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.586993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.587484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.587937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.599755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:11.608490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:11.609478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:11.610324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:11.612897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 401/775] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:11.618956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.663930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cb457-513f-452b-8f9d-4bc1d7bde114 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.704772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.705581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.705862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.706465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.707414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.708770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55e02d-34b9-49af-b166-9997880f8bdc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.716299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 375/776] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:11.717581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86d10-8e09-463d-9f60-e578f88df7d2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.717864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86d10-8e09-463d-9f60-e578f88df7d2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.718139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d86d10-8e09-463d-9f60-e578f88df7d2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.729433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 376/777] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:11.733598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.733892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.734163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.752784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:11.754002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:11.786550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1db3c9-18b1-4031-8a7d-b0d1d0deb2d4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:11.832223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 377/778] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:11.851594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:11.852462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.854413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:11.854695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:11.854972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:11.856603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:11.856916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:11.860357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:11.885588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 402/779] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:11.894801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:11.978021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc5b57-bb4c-432d-baba-cf14b4770261 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.024138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.025299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.025593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.025875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] PATH_INFO: `/v3/services/network` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.026765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.029033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:12.044072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:12.045119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:12.045991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:12.050493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 378/780] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:12.051960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.058484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afcf68-55cf-44d1-8185-eab46ff7de62 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.093786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.094612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.094942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.095215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.105590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.107095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:12.126184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:12.128416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:12.130674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:12.130674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:12.130674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:12.130674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:12.130674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 403/781] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:12.137017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.175605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bae3-9b8d-42d2-abc6-e8b36542d219 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.211289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.211922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.212202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.212526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.213229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] RBAC: Adding query filter params (name=network) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:12.213533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.214922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.219704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 379/782] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:12.238148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.251720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02062-e1b3-4cad-8c15-abe8fd11bebe None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.295471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.296143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.296427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.296702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.297419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:12.297728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.299108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.329955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 404/783] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:12.335448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.459773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ed0f-c729-44b0-98a1-c957493575d6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.480727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e100b-731f-4577-a7ba-76d4a382c4ce None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.501634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993631de-565e-48e1-95df-382893922655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.502764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993631de-565e-48e1-95df-382893922655 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.503047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993631de-565e-48e1-95df-382893922655 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.503324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993631de-565e-48e1-95df-382893922655 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.512757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993631de-565e-48e1-95df-382893922655 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.514230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993631de-565e-48e1-95df-382893922655 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-993631de-565e-48e1-95df-382893922655 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:12.531394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:12.533627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:12.535847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 405/784] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:12.542410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993631de-565e-48e1-95df-382893922655 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.545744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.546446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.546738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.547014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.547723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] RBAC: Adding query filter params (type=network) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:12.548029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.549397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.561635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 380/785] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:12.567252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.692528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993631de-565e-48e1-95df-382893922655 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.734113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e478-3a44-4772-9c10-7ff3a61c8133 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.760105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.760778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.761059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.761328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.762163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:12.762671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.764802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.781525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 406/786] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:12.786678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.800240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.800801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.801083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.801358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:12.802148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] RBAC: Adding query filter params (interface=public, service_id=423e9d6f5ee9444aa21bccb1f350310a, region_id=RegionOne) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:12.802493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:12.803922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:12.816576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 381/787] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/endpoints?service_id=423e9d6f5ee9444aa21bccb1f350310a&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:12.818892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:12.948398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ffe5-27d6-423d-b599-a824b5365970 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:12.992799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:12.993935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:12.994445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:12.994945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.006412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741ae4-ee85-4775-9bdb-3e61dca6504d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:13.011957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.013626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:13.050244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:13.051238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:13.052151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:13.052151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:13.052151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:13.052151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:13.052151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 407/788] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:13.057490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:13.066089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:13.066718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.066999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.067275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.067948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.069306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.074656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 382/789] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:12 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:13.224046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029afe0-e34c-4e65-8427-3e4f125583d6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:13.270181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:13.270836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.271092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.271366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.272101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:13.272407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.274439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.287224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 408/790] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:13.292484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:13.416775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826d128-8fa6-47d9-8849-345a028ff25e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:13.450381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:13.451194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.451474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.451744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.460695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.462977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:13.478170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:13.479130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:13.480031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:13.482489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 383/791] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:13.487843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:13.609168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14bba-973f-4eeb-90b1-9a0cdcf41e1c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:13.649203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:13.649883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.650166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.650534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.651333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:13.651640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.653064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb69ed-f04e-4094-97f2-90225998cdec None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.666148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 409/792] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:13.672181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:13.844969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359f67-057b-4c4c-b0f9-c4e0db8d33cb None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:13.849829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749a25-4959-465d-a123-548ba276eb95 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.850118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749a25-4959-465d-a123-548ba276eb95 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.850414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749a25-4959-465d-a123-548ba276eb95 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.852080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 410/793] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:13.856939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.857224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.857502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.882166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:13.883982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:13.884465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:13.884953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/2a71e33d66744c208cd0bebb9d09d521/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:13.887145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=2a71e33d66744c208cd0bebb9d09d521, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:13.888779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0ead5-a152-473d-8cde-b1e0492c0b72 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:13.892292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:13.898423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:13.908045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 384/794] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:13 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/2a71e33d66744c208cd0bebb9d09d521/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:14.020011 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 411/795] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:14.047347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74bcea-016c-4138-90a8-67345146c1c1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.047983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74bcea-016c-4138-90a8-67345146c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.048463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74bcea-016c-4138-90a8-67345146c1c1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.050117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 385/796] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:14.054442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.056713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.057188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.057467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.075792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:14.077011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:14.170090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dea1a-83eb-4146-8488-7becb9f5dd81 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.199532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 386/797] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:14.211065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.211863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.212141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.212411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.222787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.228536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:14.238422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:14.239352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:14.240199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:14.245503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 412/798] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:14.246598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.327529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398007c-4a0d-4136-a6fe-74612ec213d9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.349325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123813-8dde-4b27-8794-e55f601a6ed3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.364472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.365276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.365553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.366091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.376752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:14.383861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.384534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:14.384711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:14.386859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:14.388234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:14.389023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.389023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.389023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:14.389257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 387/799] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:14.389338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.394420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.395439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.413728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 413/800] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:14.419663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.543818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b090c2-91a3-48a5-9fef-34f6680241b3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.562543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83134-cebf-4a55-8158-3d7adaabb852 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.577567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.578558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.579066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.579568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.580521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:14.581044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.582646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f4f1f-f835-4cb9-9b9c-2972c3a5d1ce None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.592148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 388/801] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:14.594121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af545b2-eb56-4760-974d-882883ef5036 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.594673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af545b2-eb56-4760-974d-882883ef5036 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.595209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af545b2-eb56-4760-974d-882883ef5036 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.595692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.596671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.597044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.597463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.601735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 389/802] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:14.601735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af545b2-eb56-4760-974d-882883ef5036 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.612416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.613838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8b0d2f2d-762c-4334-8967-77191e58eb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:14.631519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:14.632428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:14.633929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:14.633929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:14.633929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:14.633929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:14.633929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 414/803] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:14.634876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.635157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.635431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.651814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:14.652998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:14.708633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af545b2-eb56-4760-974d-882883ef5036 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.726421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 415/804] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:14.726421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.767563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.768750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.769283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.769770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.782727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.784498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:14.809513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:14.810484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:14.811396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:14.811396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:14.811396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:14.811396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:14.813634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 390/805] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:14.814732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.820817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7c59-d086-4b77-ad57-59c5cfd2adce None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.854206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c239-984e-4660-bace-24373beff283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.854896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c239-984e-4660-bace-24373beff283 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.855172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c239-984e-4660-bace-24373beff283 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.855443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c239-984e-4660-bace-24373beff283 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.856169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c239-984e-4660-bace-24373beff283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:14.856460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c239-984e-4660-bace-24373beff283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.857820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c239-984e-4660-bace-24373beff283 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.870020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 416/806] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:14.875363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c239-984e-4660-bace-24373beff283 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:14.930776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a5a3-57e6-4dc4-93b9-3b5d13941a66 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:14.967718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:14.968637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:14.968985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:14.969345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:14.970043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:14.971529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:14.984435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:14.985418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 14:50:14.986275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:14.991171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 391/807] 10.222.0.83 () {56 vars in 1129 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:14.992175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.017944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c239-984e-4660-bace-24373beff283 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.063728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.064703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.064703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.064963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.065787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:15.066158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.067640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.083893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 417/808] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:15.083893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.110955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb462bf-cc1f-4033-afe3-c571ab2c1c0e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.163816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.165199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.165477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.165746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.173456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.174799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:15.189095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:15.190060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:15.191365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:15.195093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 392/809] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:15.195788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.198505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001f074-a4bd-4a00-aeb3-306874055b93 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.244071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.245135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.245621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.246520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.248268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:15.249174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.251923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.258498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 418/810] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:15.259556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.304909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345e79-0e74-4483-be1a-82872e465ae7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.359046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.360929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.361543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.362347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] PATH_INFO: `/v3/users/nova` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.367988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61933f-ab9e-45b8-a417-492a017008b6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.372123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.373842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:15.388614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:15.389588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:15.390557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:15.397483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 393/811] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:15.399322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.401686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.402361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.402675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.402950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.403867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:15.404173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.405589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.426504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 419/812] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:15.427716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.531084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1d73b-76c6-4c98-beb9-3afbc236e83e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.533173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae855ad-bce1-4517-91b2-ca1cdb2ac0e6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.566823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.567856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.568441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.569021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.569875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.570835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:15.571100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.571383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.571658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.571988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.572351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:15.572653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.574011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.574226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.582543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 394/813] 10.222.0.83 () {56 vars in 1124 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:15.583659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.596208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 420/814] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:15.601831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.702781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fdfc7-3716-4479-8502-90f989d6a921 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.708521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1151d-f1ad-455b-9cd0-8713e171f586 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.738110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.739167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.739652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.740144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.741332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.743005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.748544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:15.749629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:15.749915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:15.750193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:15.764229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:15.765999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:15.770026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 421/815] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:15.771401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:15.810474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:15.811468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:15.812422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:15.812422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:15.812422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:15.812422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:15.816162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 395/816] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:15.821695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:15.976973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573e1d-015e-42f0-ba52-2685668115d6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:15.977426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c8b5-f518-4df0-9a13-ba6a1544f399 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.015958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.016758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.017130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.017410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.024297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.025085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.025371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.025645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.026424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:16.026728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.028101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07a14d-846e-41e1-a219-86dd4fd05648 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.041124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 396/817] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:16.043223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d979bf-3584-452c-b9bc-79ed1e622bd4 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.043223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d979bf-3584-452c-b9bc-79ed1e622bd4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.043223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d979bf-3584-452c-b9bc-79ed1e622bd4 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.048412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 397/818] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:16.048412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d979bf-3584-452c-b9bc-79ed1e622bd4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.111712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.113382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.158072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d979bf-3584-452c-b9bc-79ed1e622bd4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.192104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.192843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.193131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.193407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.201126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.202643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1ddaef05-74b6-4f74-9cc2-de77f62889d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:16.217332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:16.218421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:16.219507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 398/819] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:16.220523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.220523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.220703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.235133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 422/820] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:16.236343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.237949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:16.239162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:16.322428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 399/821] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:16.322428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.372918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714916-871a-4f08-ba19-b7c6a0ac98ef None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.413754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.414744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.415176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.415599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.416563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:16.417020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.418665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.430726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 423/822] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:16.431753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.438082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac3958-9188-4d47-9718-0d501b80db0d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.478549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.479526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.480449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.480725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.491419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.492851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:16.510996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:16.511911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:16.512851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:16.512851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:16.512851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:16.512851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:16.512851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 400/823] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:16.513154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.538808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8ac2d-abf4-4371-9cd6-cc8e539d2db7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.571991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.572771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.573048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.573322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] PATH_INFO: `/v3/services/network` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.573959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.575342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a272ba10-0906-47be-9add-531a00cbd389 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:16.587130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:16.588139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:16.588996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 424/824] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:16.589793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af2ec9-0686-46bd-a553-79268c3bf472 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.589869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af2ec9-0686-46bd-a553-79268c3bf472 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.590479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af2ec9-0686-46bd-a553-79268c3bf472 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.594551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 425/825] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:16.594551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af2ec9-0686-46bd-a553-79268c3bf472 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.616920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130271-b4fa-413f-9227-57f2482aaa45 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.650203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.651159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.651436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.651710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/7fc10eb4012647b9874edca4482f7c14/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.666555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=7fc10eb4012647b9874edca4482f7c14, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.668009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.686508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 401/826] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:16 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/7fc10eb4012647b9874edca4482f7c14/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:16.687495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.699849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af2ec9-0686-46bd-a553-79268c3bf472 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.736964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.737644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.737931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.738207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.738981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:16.739287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.740975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144624-5d8c-4f6d-95f3-fd766c492f1a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.754531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 426/827] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:16.755783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.756071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.756346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.774071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:16.775395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:16.804370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c525-3503-4f64-afbc-bf967d0215e7 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.840886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:16.841991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:16.842520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:16.843095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:16.844192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] RBAC: Adding query filter params (name=network) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:16.844710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:16.845107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 427/828] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:16.846974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:16.851423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.853569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 402/829] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:16.859775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:16.984069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740cfdb-1b1a-45a5-96fa-ceaea592dab9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:16.985709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a3287-72e7-4dd1-90d1-780cdd8a564d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.024985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.025576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.025848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.026122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.026772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.026900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] RBAC: Adding query filter params (type=network) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:17.027151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.027732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.028081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.028422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.028528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.036551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 403/830] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:17.036805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.038114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.041980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:17.052518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:17.053614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:17.055699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 428/831] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:17.059163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.149805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584e5d4-b4a9-40f2-a615-8096e0d2c705 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.169009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54344d5-5723-4565-a2ef-2db6991af254 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.196443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.197249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.197532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.197806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.208775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.209442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.209999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.210288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.210502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:17.211905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.213317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:17.219804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:17.221694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 404/832] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:17.223204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.238912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 429/833] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:17.239746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.395281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807d449-5937-442f-bb89-3e27ec9704cc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.436735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.437407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.437686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.437960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.438781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:17.439397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.440520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.452935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e8b78-0a6e-4ad4-aee2-250b57da4345 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.454442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 405/834] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:17.460717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.494784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.495459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.495753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.496046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.496794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:17.497097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.498498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.507107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 430/835] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:17.516463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.582951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f28ab-4659-4d7b-b41c-262719fd497d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.616759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.617962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.618241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.618541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.619369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:17.619733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.621218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.623981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbe2bf-ac83-40a7-9b74-21948ff52cdd None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.627684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 406/836] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:17.664676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.665415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.665690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.665963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.678129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.680221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:17.698738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:17.700931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:17.702703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:17.702703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:17.702703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:17.702703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:17.702703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 431/837] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:17.708414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:17.825919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e9a0-6860-4ecb-b1ae-6b036318ea3b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:17.861515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:17.862180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:17.862492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:17.862772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:17.863494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:17.863797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:17.865158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:17.877849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 407/838] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:17.883665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.001650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171262a-04dd-4813-9be3-7aaf786f77d7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.035572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.036380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.036664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.036935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] PATH_INFO: `/v3/users/swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.045237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.046746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:18.062985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:18.064020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:18.065456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 432/839] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:18.070489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.172469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1e9bc-dc09-4cd2-98ff-a4174deb4d0f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.208204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.209173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.209653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.210100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.211173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:18.211654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.213243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d4a1-f781-4fea-ad0f-591c74b10cc5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.230969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 408/840] 10.222.0.83 () {56 vars in 1135 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:18.241571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.251921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b5f-96d7-4b2a-b452-2f8528a8af86 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.252205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b5f-96d7-4b2a-b452-2f8528a8af86 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.252480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b5f-96d7-4b2a-b452-2f8528a8af86 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.254061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 409/841] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:18.259510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.260074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.260074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.279567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:18.281636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:18.354572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f34e0-72df-42e9-9d53-0fdcaee9822b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.366508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 410/842] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:18.386758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.393842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.394528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.394811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.395080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.468161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.469262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.490110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5df217-41b8-4cba-b68a-6128cb7ce844 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.525793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.526797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.527174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.527565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.528314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.529771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:18.542325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:18.543575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:18.544612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:18.550425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 411/843] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:18.552036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.576226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 433/844] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:18.581278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.677749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3eb1-2f5f-407b-bba8-70e729000f2a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.678154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38ad6-7f88-48cf-b833-0cbda6d40c31 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.720489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.720674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.721160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.721446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.721719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.722327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.722534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:18.722789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.723209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.723670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.724158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.733654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 412/845] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:18.734682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.738942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.740385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:18.763842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:18.765051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:18.765924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:18.765924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:18.765924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:18.765924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:18.768654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 434/846] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:18.774225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:18.851065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfc44f-9e71-481c-a210-4d545de003cc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.873016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4cbe-3b57-4ae4-a58a-e3d6e0a3fe71 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:18.885216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.886196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.886708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.887182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.888068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:18.888566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.890200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.901096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 413/847] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:18.911816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:18.912586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:18.912869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:18.913669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:18.915026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:18.915334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:18.916754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:18.932272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 435/848] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:18.937966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.050853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6613dc-357d-44ba-bc69-bd8d9c51870a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.087453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.088618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.089119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.089586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.098113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.099597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-95c02354-646d-4dd0-aa46-c16dc70b039c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:19.115490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:19.116688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:19.117627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 414/849] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:19.124479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.138285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4363f6-99c6-4276-a4b3-3127fdcb07c9 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.138598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4363f6-99c6-4276-a4b3-3127fdcb07c9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.138986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4363f6-99c6-4276-a4b3-3127fdcb07c9 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.141797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 415/850] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:19.145394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.145678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.145953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.162774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:19.163991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:19.238534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 416/851] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:19.245322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509b78-4efb-4ebe-969c-c7412ac8ce51 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.255849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.288760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.290214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.290690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.291112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.292059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:19.292514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.294074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.307248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 436/852] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:19.313107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.357834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe6b8-d4dc-42ed-94bd-b07f3da9977a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.392365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.393204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.393488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.393761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.404424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:19.412208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:19.413373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:50:19.415011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:19.416378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8b318-702d-4143-bf59-83e41d1dcced None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.419468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 417/853] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:19.419987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.451502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.452427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.452884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.453309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.454289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=2a71e33d66744c208cd0bebb9d09d521) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:19.454772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.456354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.470546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 437/854] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/role_assignments?user.id=2a71e33d66744c208cd0bebb9d09d521&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:19.529628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303d56-b7e6-4c47-bdff-34d56c97b951 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.566878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.567469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.567748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.568019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.568767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:19.569068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.570725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.584779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 418/855] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:19.587731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.691353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7786af-c815-4ab9-a315-0f7ef442f8fc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.723908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.724698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.724979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.725246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.737228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.738688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:19.756212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:19.757173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:19.758119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:19.758119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:19.758119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:19.758119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:19.761968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 438/856] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:19.763333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:19.868132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31bc8-bde8-4e31-81d1-71be0ce4fe41 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:19.903822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:19.904582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:19.904863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:19.905134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:19.905840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:19.906523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:19.907763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07552d6b-ea77-404f-b540-23a6682f0ecb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:19.924807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 419/857] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:19.926185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.029647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4780-ba25-4953-b0ea-e1147d4ab08f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.048509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e346-2ff0-4f2b-9e0a-e6a16048618f None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.048810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e346-2ff0-4f2b-9e0a-e6a16048618f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.049056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e346-2ff0-4f2b-9e0a-e6a16048618f None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.050561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 420/858] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:20.056022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.056290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.056563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.070024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.071345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.071837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.072339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.077286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:20.078517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:20.081436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.083135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:20.097957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:20.098928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:20.099865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:20.102942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 439/859] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:20.108548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.172139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 421/860] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:20.194661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.225525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbad8b8-dee7-42ff-9bba-9c16b5bd1e8e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.270341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.271476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.271761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.272031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.273040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:20.273704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.275453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.292327 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a48a-731d-4203-a493-08d8e26b63c8 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.302447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 440/861] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:20.303887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.330690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.331687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.332064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.332436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.333269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.334777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:20.348305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:20.350129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:20.352715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:20.362409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 422/862] 10.222.0.83 () {56 vars in 1132 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:20.362409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.424460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c41232-068e-4ac4-aee0-9fed10b636ad None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.458817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b915-5348-4119-881b-3f562fd7afe0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.465033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.465679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.466362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.466668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.494066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.494893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.495241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.495616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.496385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:20.496757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.498243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.507882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 423/863] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:20.508802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.544699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.546222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.627286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b14c8-a6ec-44b9-b03c-a662cac90c2d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.640620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 441/864] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 14:50:20.646193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.668700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.669519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.669901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.670274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.671098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:20.671503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.672962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.686949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 424/865] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:20.757508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531680b-8823-4d9b-b393-79342014e167 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.795009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.795815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.796092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.796361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.807478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.808943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:20.828214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:20.830076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:20.831975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:20.831975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:20.831975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:20.831975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:20.834091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 442/866] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:20.835523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:20.935484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2571-503c-48df-a14e-9dd0831afc16 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:20.968531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:20.969164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:20.969446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:20.969714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:20.970508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:20.970813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:20.972207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:20.988360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 425/867] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:20.989816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:21.102020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484b251-b46f-443d-8bfb-88293244ce25 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:21.137270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:21.138070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.138352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.138655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.145299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:21.146764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8f2a2079-e496-4ce0-8506-94fd8cf19e33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:21.163289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:21.165561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:21.168776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 443/868] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:21.172821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:21.233288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b9ba-5719-46b0-886d-5aa2d954db8f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.233601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b9ba-5719-46b0-886d-5aa2d954db8f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.233767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468b9ba-5719-46b0-886d-5aa2d954db8f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.235489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 444/869] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:21.241355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.241614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.243306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.260777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:21.262250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:21.292146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb690be-e145-4485-a6d3-64d2b9acca25 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:21.328969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 445/870] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:21.339257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:21.340241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.340723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.341247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.342442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:21.343271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:21.346325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:21.353432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:21.366191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 426/871] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:21.375334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:21.460377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e487a-de71-4106-af5d-c31fe018989a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:21.498001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:21.499049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.499531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.500010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.500944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:21.502511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450266f-0849-4eb3-948c-54e2320e20e8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:21.505100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb1dc-9e75-4324-baae-4edf9dd3a485 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:21.516562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 446/872] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:21 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:21.560804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:21.561875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.562466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.562940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.563981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=7fc10eb4012647b9874edca4482f7c14) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:21.564513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:21.566241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee5b74-0a49-4a3c-bfc5-f63d2ee8e9b5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:21.578560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 427/873] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:21 2022] GET /identity/v3/role_assignments?user.id=7fc10eb4012647b9874edca4482f7c14&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:21.862424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b641-3831-4fbc-8ceb-2fcd5cf04395 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.863396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b641-3831-4fbc-8ceb-2fcd5cf04395 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.863965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b641-3831-4fbc-8ceb-2fcd5cf04395 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.865533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 447/874] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:21.871566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:21.872083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:21.872161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:21.891052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:21.892597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:21.963013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 428/875] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:21.985775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1b641-3831-4fbc-8ceb-2fcd5cf04395 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:22.093637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1b641-3831-4fbc-8ceb-2fcd5cf04395 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:22.127893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:22.128609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:22.128890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:22.129158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:22.129798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:22.131175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:22.143047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:22.144044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:22.144901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 448/876] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:22.150832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:22.253090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6fab-fb23-46ed-9940-0e4ef3fbf2d5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:22.290604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:22.291274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:22.291554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:22.291829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:22.292521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:22.292824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:22.294186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15d3dc-4b3a-4358-9fef-293a9e996238 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:22.299307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 429/877] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:22.307477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:22.406502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c0984-f387-4932-af58-19e58e3fc48c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:22.440666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:22.441332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:22.441611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:22.441937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:22.442675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:22.443022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:22.444366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105d855-420b-4fa3-b90d-02d9ddf89852 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:22.449909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 449/878] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:23.013245 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fdb2c-e7a5-4b06-960b-bbe369b51013 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:23.013773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fdb2c-e7a5-4b06-960b-bbe369b51013 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:23.013773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fdb2c-e7a5-4b06-960b-bbe369b51013 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:23.015461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 430/879] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:23.020706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997880b-0641-4cec-b697-1c1225ad4184 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:23.021147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997880b-0641-4cec-b697-1c1225ad4184 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:23.021272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997880b-0641-4cec-b697-1c1225ad4184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:23.037959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c997880b-0641-4cec-b697-1c1225ad4184 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:23.039681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c997880b-0641-4cec-b697-1c1225ad4184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:23.103231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 450/880] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:23.126002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fdb2c-e7a5-4b06-960b-bbe369b51013 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:23.235176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fdb2c-e7a5-4b06-960b-bbe369b51013 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:23.271261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:23.271930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:23.272212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:23.272487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:23.273246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:23.274644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25968d78-18cc-49ec-bfc4-fbb03c115a37 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:23.293057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 431/881] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:23 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:23.843561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af5388-d6d9-48aa-b06a-1a05455c5abd None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:23.843561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af5388-d6d9-48aa-b06a-1a05455c5abd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:23.843561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af5388-d6d9-48aa-b06a-1a05455c5abd None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:23.844470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 451/882] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:23.850305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd531dc-c40f-429a-97a3-a7284d2428fa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:23.850971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd531dc-c40f-429a-97a3-a7284d2428fa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:23.851489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd531dc-c40f-429a-97a3-a7284d2428fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:23.869615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4fd531dc-c40f-429a-97a3-a7284d2428fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:23.871272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4fd531dc-c40f-429a-97a3-a7284d2428fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:23.955116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 432/883] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:23.977610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af5388-d6d9-48aa-b06a-1a05455c5abd None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.058382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c054-e62c-4f54-8013-63dceabcdbb2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.059387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c054-e62c-4f54-8013-63dceabcdbb2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.059862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c054-e62c-4f54-8013-63dceabcdbb2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.061638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 433/884] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:24.066878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.067408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.067887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.085444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af5388-d6d9-48aa-b06a-1a05455c5abd None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.087053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:24.088465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:24.120866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.121726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.122029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.122276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.123035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.124459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:24.136895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:24.137958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:24.138834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:24.142845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 452/885] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:24.144868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.177985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 434/886] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:24.199098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.270499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a0b20-595e-472a-89fc-a1b883fd65a8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.305704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1107-1920-44e1-bcab-e36b811e495f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.307810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.309501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.309997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.310518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.311417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:24.311921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.313896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.324811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 453/887] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:24.326912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.346872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.348019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.348600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.349020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.361710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:24.372974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:24.374058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:24.374927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:24.378529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 435/888] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:24.381286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.448021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4fc0-d3a7-48e5-aaba-e9559781e4ce None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.494961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.495966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.496253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.496531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.497529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:24.497840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.499227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.499594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72603635-b407-4919-aa94-e0e5fcb18ffc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.514179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 454/889] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:24.515937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.550425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.551176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.551457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.551732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.552647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:24.552960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.555771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.572253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 436/890] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:24.573574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.627013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9bc-f1ef-4c37-b91c-07fa2515a177 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.659632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.660660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.661153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.661640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.662665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] RBAC: Adding query filter params (interface=public, service_id=6aafb7c924a3438a959c6504ad6ee8b5, region_id=RegionOne) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:24.663189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.665297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.673300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 455/891] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/endpoints?service_id=6aafb7c924a3438a959c6504ad6ee8b5&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:24.675153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07c146-9316-4dde-b02b-2efa13e6e48c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.678788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.711925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.712727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.713011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.713290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.724433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.725872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-1e35dcd0-feb6-477d-bd98-8249636e5d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:24.743655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:24.744656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 437/892] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ed32a-0a30-4955-927b-3001bc643378 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ed32a-0a30-4955-927b-3001bc643378 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ed32a-0a30-4955-927b-3001bc643378 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.750406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 438/893] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:24.752695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ed32a-0a30-4955-927b-3001bc643378 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.785328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd22583-ebfa-485f-b69c-feb6bf68ee98 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.818946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.819699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.820012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.820638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.820965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.822322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b0cbc-99d5-4e82-b240-147b6a50af9f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.834729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 456/894] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:50:24.834729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.834729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.834729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.857126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:24.858526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:24.866736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ed32a-0a30-4955-927b-3001bc643378 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:24.905552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:24.906146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:24.906454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:24.906787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:24.910120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:24.910120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:24.910120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:24.930549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 439/895] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:24.930549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:24.932474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 457/896] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:24.959970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.048723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687e9a-b888-4624-b8e3-744d080bd4c0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.080563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163b2e-b8a3-4063-9740-ecb5a329f55f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.089826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.090677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.091317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.091602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.102149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.103936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.116454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.117156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.117440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.117712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.118467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.119813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:25.121900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:25.122897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:25.123856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 440/897] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:25.130442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.136310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 458/898] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:24 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:25.241643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03084e-9eb2-4766-a7ba-b1b63e980bfc None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.283764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.284440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.284718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.284988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.285821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:25.286123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.287594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.306821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 441/899] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:25.312170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.423023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a3c44-467a-45ad-8b9b-94f8da9718ca None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.459427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.460114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.460389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.460733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.531619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.532987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28374-229c-463c-94d2-29e510c42dc3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.630967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 459/900] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:25.637215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.639424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f6e8b-a2c7-48b3-84bb-e6ba50da2d51 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.639708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f6e8b-a2c7-48b3-84bb-e6ba50da2d51 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.639982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f6e8b-a2c7-48b3-84bb-e6ba50da2d51 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.642438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 460/901] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:25.647675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.647957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.648272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.664329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:25.665495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:25.738755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 461/902] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:25.754253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.755141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc0f6a-a975-4d09-a1e8-97a5ebcd0b77 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.797517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.799189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.799706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.800381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.811999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.813550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:25.832178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:25.833316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:25.834467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:25.834467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:25.834467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:25.834467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:25.840281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 442/903] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:25.841256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.864089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769acba6-4de1-4720-89bd-f077cf54a441 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:25.897728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:25.898643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:25.898922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:25.899237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:25.899886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:25.901363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:25.914592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:25.915642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:25.916532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:25.921096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 462/904] 10.222.0.83 () {56 vars in 1132 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:25.922352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:25.968190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d089-a0c6-4470-9bd2-70b2b757ed1d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.016782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.017800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.018443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.018879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.020309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.020775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.023121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.024914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c65d1-c961-460d-b8f3-4bd1fe70837a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.043237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 443/905] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:26.047936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.069051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.069767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.070047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.070319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.071063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.071580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.072770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.082483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 463/906] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:26.083870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.165318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1af84f-2253-4991-b385-014f52dcdc64 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.197608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.198482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.198923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.199076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.202268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1936d-e5c7-4b1b-be20-9fa4fd29d527 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.205752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.207242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:26.221596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:26.222583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:26.223508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:26.226955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 444/907] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:26.229260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.244102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.245279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.245767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.246681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.247894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.248575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.250475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.257527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 464/908] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:26.263930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.334573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5e32-5d23-4253-b65f-d7af22fd7649 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.369598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.370669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.371292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.371721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.372947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.373421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.375481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.389729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbb1b-6960-43af-b78a-a22f921476d7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.391839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 445/909] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:26.397646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.430951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.431918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.432383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.432845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.433819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] RBAC: Adding query filter params (interface=public, service_id=85bc4586a01048c592d1ccdcdd7ccddc, region_id=RegionOne) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.434346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.436000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.442039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 465/910] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/endpoints?service_id=85bc4586a01048c592d1ccdcdd7ccddc&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:26.447991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:26.509366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eff08-55fc-4e43-a19b-495f10f97eb6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.544514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.545606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.546113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.546643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.547763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] RBAC: Adding query filter params (role.id=ca9dcb6ceb7740b8a98db73ff59974d9, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=7fc10eb4012647b9874edca4482f7c14) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:26.548287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.549916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55351765-bcff-4f9e-9966-b4bac7866994 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.558568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 446/911] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/role_assignments?user.id=7fc10eb4012647b9874edca4482f7c14&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=ca9dcb6ceb7740b8a98db73ff59974d9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:26.580190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a690-9987-4d87-8027-11f26b892671 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:26.638534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:26.639663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:26.640194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:26.640686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:26.641577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:26.643185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28d201-f99f-47d3-89d3-420c28035dd2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:26.655342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 466/912] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:26 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:50:27.160515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b7d5-38df-45fa-95af-1e9267860103 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.161016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b7d5-38df-45fa-95af-1e9267860103 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.161195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b7d5-38df-45fa-95af-1e9267860103 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.162753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 447/913] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:27.168504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.168729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.169012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.187212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:27.188789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:27.269173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 467/914] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:27.307378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769b7d5-38df-45fa-95af-1e9267860103 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.416169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769b7d5-38df-45fa-95af-1e9267860103 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.451906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.452624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.452903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.453174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.453816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.455204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-dc6c00ee-4966-4a81-b7d7-40c45bfc7999 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:27.467908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:27.468975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.469820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 448/915] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:27.479473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.502808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380d1e-67ba-48c4-b85d-77da1265f61d None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.503444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380d1e-67ba-48c4-b85d-77da1265f61d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.503955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380d1e-67ba-48c4-b85d-77da1265f61d None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.505638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 449/916] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:27.511481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.511991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.512448 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.531713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:27.533082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:27.609961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 450/917] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:27.632983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.639930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a53373-11db-4c64-8471-d0498fe0fc9b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.674904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.675639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.675921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.676191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.676887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:27.677189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.678700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.685007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 468/918] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:27.691167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.759746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18066b2-7d31-4c7a-aae2-1073e435b6fa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.794436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2469c-4ee2-4e1a-8cb4-1fd58cf487a4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.796248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.797292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.797780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.798286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.799181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.800743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:27.813688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:27.814777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:27.815630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:27.823760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 451/919] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:27.824835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.829863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.830158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.830486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.830769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.831659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:27.831974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.833429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.845021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 469/920] 10.222.0.83 () {56 vars in 1114 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:27.846045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:27.944473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddeee7-34de-4744-9900-aa4613027569 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.956947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f89c6-772e-435e-b575-e69cad532162 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:27.980360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.981417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.981906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.982401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.983316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:27.983859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.985444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.991427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:27.992713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:27.997579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:27.997579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:27.997579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:27.997579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:27.998264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 452/921] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:27.999546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:28.026834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 470/922] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:28.119895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcfb25-f7db-4039-afb0-aa61ff5c7442 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:28.164813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:28.165966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:28.166521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:28.167068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:28.168117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:28.168689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:28.170642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:28.177631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 453/923] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:28.185509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:28.294165 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84ed29-0c75-49da-bca2-5d170c266525 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:28.328253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:28.328924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:28.329205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:28.329477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:28.330265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] RBAC: Adding query filter params (interface=public, service_id=31df8cfe29af468bb3a12b5ddc800cf0, region_id=RegionOne) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:28.330603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:28.331974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00064ad-deb1-4465-be10-141bac09a203 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:28.337804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 471/924] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:28 2022] GET /identity/v3/endpoints?service_id=31df8cfe29af468bb3a12b5ddc800cf0&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:28.343198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:28.447533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b0a9-e601-4a8b-bcf1-ca12a8337b6c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:28.483240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:28.484286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:28.484775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:28.485358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:28.486237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:28.487827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5100cdb-3614-4b91-ba85-1e988b93a3c2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:28.495373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 454/925] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:28 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:50:28.991647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ec8a-4988-4a29-b52c-65743f0b3cd6 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:28.992335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ec8a-4988-4a29-b52c-65743f0b3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:28.992822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ec8a-4988-4a29-b52c-65743f0b3cd6 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:28.994693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 472/926] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:29.000036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.000578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.001066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.018200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:29.019801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:29.030247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7ca9-72d5-4eff-a7b0-532409ebc167 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.030881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7ca9-72d5-4eff-a7b0-532409ebc167 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.031375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7ca9-72d5-4eff-a7b0-532409ebc167 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.033032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 473/927] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:29.038251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.038818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.039311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.056388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:29.057846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:29.098417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 455/928] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:29.116350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.143153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 474/929] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:29.166019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.225446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8ae2-5d01-4355-aa94-2611c287354e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.261657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.262873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.263379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.263874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.265436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f6161-b126-4001-b422-69a73f18b975 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.275216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:29.283761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:29.284713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:29.285567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:29.288265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 456/930] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:29.294970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.306310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.308203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.308652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.309076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.310244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.312408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:29.328918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:29.329963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 14:50:29.330856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:29.337075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 475/931] 10.222.0.83 () {56 vars in 1132 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:29.338735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.422072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f6e9-5535-455f-9005-25749becd4b2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.461992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.462685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.463163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.463347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbce54-cb2a-4dea-9731-46fef444d2d7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.463731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.464605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:29.465215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.467531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.482545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 457/932] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:29.484164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.508721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.509738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.510688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.511168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.512301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:29.512955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.515289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.522076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 476/933] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:29.528568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.590126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2752dc-d96f-4a07-9e4f-8fe3ed4996bd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.623645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.624658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.625171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.625645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.638072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.639961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.652015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d874f-7948-4c0a-95c8-364d51d0290d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:29.658607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:29.659502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:29.660410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:29.660410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:29.660410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:29.660410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:29.667501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 458/934] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:29.668637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.690456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.691115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.691397 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.691677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.692861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:29.693170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.694921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.706019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 477/935] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:29.707730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.771301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7290-9290-4f44-b4a9-5e9d08e37320 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.805375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.806033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.806311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.806443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804177-65f0-4da8-808d-f3510a92b216 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.806627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.807378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:29.807678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.809102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.821195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 459/936] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:29.826496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:29.843528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.844195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:29.844466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:29.844738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:29.845473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:29.846859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:29.874434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 478/937] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:29.954815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a961f-fa9c-46a1-8a89-9754d4638b88 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:29.998982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:29.999805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.000090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.000366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.008221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.009680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:30.025849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:30.026895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:30.027867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 460/938] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:30.033015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.149363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a73-5cce-45a9-9b01-48fcc023a80f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.184959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.185656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.185940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.186221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.187112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:30.187426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.188901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3881d5-5b67-48e1-bc70-fcf2edbe7ebf None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.208551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 479/939] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:30.214610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.314123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79f5-b4bf-4827-a926-ca7427571638 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.348977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.349778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.350054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.350321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.361429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.362901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.370227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3479ee-273f-4e69-99cf-2b3239ee82bc None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.370749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3479ee-273f-4e69-99cf-2b3239ee82bc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.370857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3479ee-273f-4e69-99cf-2b3239ee82bc None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.375770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 480/940] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:30.377620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.378103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.378596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:30.381496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:30.382461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:30.383750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:30.383750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:30.383750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:30.383750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:30.383750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 461/941] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:30.388617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.401091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:30.402418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:30.475357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 481/942] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:30.498723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.502427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4027bd-1af8-450c-982f-cb5d633eb30e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.537413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.538077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.538429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.538746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.539523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:30.539832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.541363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.554435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 462/943] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:30.558776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.608444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efd01c-d855-4cc2-8606-8fb023cf1fe6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.643906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.644667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.645017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.645366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.646139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.647891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.649668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING keystone.server.flask.common [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 10 14:50:30.667423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e0fd-5780-4079-b9de-104df3fae6ea None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.667987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.sql.core [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 14:50:30.667987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 14:50:30.667987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [parameters: {'id': '542773769f6f459c805fdbee2c57f27e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 14:50:30.667987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79936) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 542773769f6f459c805fdbee2c57f27e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 542773769f6f459c805fdbee2c57f27e. May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application conn.query(q) May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:50:30.676068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result.read() May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 10 14:50:30.676907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return project_ref.to_dict() May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application next(self.gen) May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application yield resource May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application next(self.gen) May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.session.rollback() May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.force_reraise() May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise self.value May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application session.commit() May 10 14:50:30.677881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._prepare_impl() May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.session.flush() May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._flush(objects) May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application compat.raise_( May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application flush_context.execute() May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 14:50:30.678819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rec.execute(self) May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 14:50:30.680836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application conn.query(q) May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application result.read() May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:50:30.682836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [parameters: {'id': '542773769f6f459c805fdbee2c57f27e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.684822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:30.687014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.Conflict( May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 542773769f6f459c805fdbee2c57f27e. May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 482/944] 10.222.0.83 () {60 vars in 1140 bytes} [Tue May 10 14:50:30 2022] POST /identity/v3/projects => generated 262 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:30.689174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.713715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.714887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.715378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.715900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.726871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:30.745905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:30.746926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:30.747834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:30.750339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 463/945] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:30.756013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.798261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276c908-a7ee-4196-ab84-1eb6706c0e87 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.838547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.839505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.839885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.840260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.847090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.848847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.858881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf31b6-0c32-4ec5-bdab-3518bb85467e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-22299e76-0879-4ebb-933d-5b90ce5e7074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:30.864150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:30.865197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:30.866573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:30.872695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 483/946] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:30.872695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124fc86-65da-4f77-8e8e-75abfbd8d6b9 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.872695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124fc86-65da-4f77-8e8e-75abfbd8d6b9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.872695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124fc86-65da-4f77-8e8e-75abfbd8d6b9 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.873952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 484/947] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:30.874636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124fc86-65da-4f77-8e8e-75abfbd8d6b9 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:30.903890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:30.904848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.905133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.905670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.909585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:30.909903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:30.912141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672970e5-5519-4e39-8566-46ef4b1fe6a3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:30.928090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 464/948] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:30.929402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:30.929695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:30.929976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:30.946094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:30.947303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:30.978054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124fc86-65da-4f77-8e8e-75abfbd8d6b9 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.009110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 465/949] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:31.014580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.014697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.014784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.037056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 485/950] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:31.037981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:31.132824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18260ac4-2e7b-42b6-b951-2376de8cd676 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.153629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902bc4-d3a8-481e-8c81-24809821d524 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.169787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.170902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.171257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.171542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/7fc10eb4012647b9874edca4482f7c14/roles/ca9dcb6ceb7740b8a98db73ff59974d9` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.173945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=7fc10eb4012647b9874edca4482f7c14, role_id=ca9dcb6ceb7740b8a98db73ff59974d9)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.175528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.193607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.194754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.195714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.196133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.197016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.198510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 466/951] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:31 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/7fc10eb4012647b9874edca4482f7c14/roles/ca9dcb6ceb7740b8a98db73ff59974d9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:31.198853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:31.213354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:31.214718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:31.215775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:31.218573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 486/952] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:31.224891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:31.337211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86aac3-b2d9-4fc3-bee7-103ddf6ce3a2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.373564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.374428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.374805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.375175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.375979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:31.376380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.377928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.384007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 467/953] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:31.389498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:31.499913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cab91-51a9-408c-8cf0-6f4b293568f0 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.533372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.534043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.534325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.534624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.535328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:31.535636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.536994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cdeca-2167-4140-82ae-c0abfb2def90 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.542258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 487/954] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:31.548133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:31.647616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d03e-ca7d-4907-a077-3780a627bcde None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:31.682153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:31.683272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:31.683780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:31.684255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:31.685209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:31.686810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da4668-a1b6-4fff-8683-66743e1f0ba4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:31.708368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 468/955] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:32.345605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ff9bb-7de1-4ae6-b165-d2327979ead0 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:32.346077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ff9bb-7de1-4ae6-b165-d2327979ead0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:32.346241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ff9bb-7de1-4ae6-b165-d2327979ead0 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:32.348096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 488/956] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:32.353974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:32.354322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:32.354575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:32.376920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:32.378089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:32.469978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 469/957] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:32.494176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ff9bb-7de1-4ae6-b165-d2327979ead0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:32.619073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ff9bb-7de1-4ae6-b165-d2327979ead0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:32.657016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:32.658154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:32.658637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:32.659106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:32.659926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:32.661442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:32.676954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:32.677986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:32.678876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:32.686005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 489/958] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:32.687504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:32.790839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19063ce-30a4-4bf1-bc66-5b1d1380d8fa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:32.825478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:32.826346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:32.826792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:32.827205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:32.828041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:32.828508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:32.830007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710bc65-ede7-40d0-9eb7-480114a676b3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:32.838482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 470/959] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:32.840905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:32.942057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec925c-d020-4f8d-bf75-b0cbb76df057 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:32.978095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:32.978818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:32.979101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:32.979375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:32.980076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:32.980380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:32.981736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71580cd4-942b-458f-b8cb-c8d5f23669aa None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:32.990502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 490/960] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:33.482648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68571-527c-43a8-893d-fa139904d4ca None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.483009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68571-527c-43a8-893d-fa139904d4ca None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.483207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68571-527c-43a8-893d-fa139904d4ca None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.484865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 471/961] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:33.490402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2a3e-8441-4f52-b911-195cd29de72f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.490701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2a3e-8441-4f52-b911-195cd29de72f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.491248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2a3e-8441-4f52-b911-195cd29de72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.508022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e7ed2a3e-8441-4f52-b911-195cd29de72f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:33.509516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e7ed2a3e-8441-4f52-b911-195cd29de72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:33.576988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 491/962] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:33.592997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c3f3-8a1d-4b7e-aca4-ce2556428a32 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.593265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c3f3-8a1d-4b7e-aca4-ce2556428a32 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.593550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c3f3-8a1d-4b7e-aca4-ce2556428a32 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.595192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 472/963] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:33.600667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.600956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.601800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.606841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243c3f3-8a1d-4b7e-aca4-ce2556428a32 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:33.619191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:33.620353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:33.686128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 492/964] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:33.708659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243c3f3-8a1d-4b7e-aca4-ce2556428a32 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:33.709330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:33.756459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:33.759186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.759469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.759744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.760836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:33.761897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:33.763247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING keystone.server.flask.common [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 10 14:50:33.779489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.sql.core [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 14:50:33.779489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 14:50:33.779489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [parameters: {'user_id': '320ee4896e7e4dec96484eb4e03e7f1b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 14:50:33.779489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79937) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application conn.query(q) May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application result.read() May 10 14:50:33.801417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:50:33.802411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application next(self.gen) May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application yield resource May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application next(self.gen) May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self.session.rollback() May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self.force_reraise() May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise self.value May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application session.commit() May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 14:50:33.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._prepare_impl() May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self.session.flush() May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._flush(objects) May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application compat.raise_( May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application flush_context.execute() May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rec.execute(self) May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 14:50:33.805458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application result = connection._execute_20( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 14:50:33.806336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application result = self._query(query) May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application conn.query(q) May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application result.read() May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 14:50:33.808653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [parameters: {'user_id': '320ee4896e7e4dec96484eb4e03e7f1b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:33.810418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 10 14:50:33.812171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 473/965] 10.222.0.83 () {60 vars in 1131 bytes} [Tue May 10 14:50:33 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:33.814852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:33.867341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70d22-1dc4-4076-a08c-7e49ed1bce1b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:33.901456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:33.902436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.902796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.903176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.915865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:33.924080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64665b2f-ddd7-470a-a016-72d3360d6cff None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:33.924761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:33.925676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:33.926740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 493/966] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:33.938536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:33.967773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:33.968546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:33.968831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:33.969108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:33.977180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:33.978895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:33.994422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:33.995399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 14:50:33.996357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:34.000607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 474/967] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:34.001536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.043572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bc266-ae33-405e-8f26-cda0d4a5e273 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.076952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529a17-f10b-423f-a122-21816701efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.077960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529a17-f10b-423f-a122-21816701efff None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.078477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529a17-f10b-423f-a122-21816701efff None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.078951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529a17-f10b-423f-a122-21816701efff None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.079923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529a17-f10b-423f-a122-21816701efff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.080431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529a17-f10b-423f-a122-21816701efff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.081981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529a17-f10b-423f-a122-21816701efff None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.090886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 494/968] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:34.096487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529a17-f10b-423f-a122-21816701efff None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.107421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bb84-a4bf-4baf-8050-ee2ddd0d583c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.147310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.148536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.149045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.149550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.151158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.151720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.154715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70a82d-0f58-4545-9a94-22e08d158831 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.195196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 475/969] 10.222.0.83 () {56 vars in 1101 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:34.196584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22799b4-5028-4835-959c-fe65cd2b70b1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.196872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22799b4-5028-4835-959c-fe65cd2b70b1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.197150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22799b4-5028-4835-959c-fe65cd2b70b1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.198603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 476/970] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:34.204215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.204526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.204776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.211894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529a17-f10b-423f-a122-21816701efff None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.226439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:34.226439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:34.249461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.250602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.251026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.251438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.264302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.266056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:34.285395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:34.286449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:34.287531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:34.287531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:34.287531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:34.287531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:34.290285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 495/971] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:34.295717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.326546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 477/972] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:34.352517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.414203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d80a-81b1-4590-b51a-d56dcc1d0b46 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.456559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.457387 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.457521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.457774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.458999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.458999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.460165 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.475015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 496/973] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:34.481250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.485412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd0978-5149-4f2b-9e64-9dbf62d3974f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.521572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.522373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.524918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.524918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.524918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.526554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:34.538269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:34.540201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:34.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:34.543834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 478/974] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:34.549442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.594799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c026c-3713-4aba-9ca7-1478c9f2a98c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.626647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.627590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.627982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.628354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.635739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.637263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.645107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86da33-eaab-4392-97c4-c872bca93808 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:34.652880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:34.654075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 14:50:34.655365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:34.661246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 497/975] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:34.662201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.678968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.679694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.680067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.680343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.681046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.681356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.682758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.692395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 479/976] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:34.693322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.758271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642dfc-9e23-48e1-9543-0d3cb6cf845d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.790075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5900-487d-4d16-8a18-2ccdd06027b4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.792661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.793332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.793616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.793890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.794754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.795062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.796498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.813874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 498/977] 10.222.0.83 () {56 vars in 1149 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:34.819068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.829018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.829676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.829956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.830225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.830957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.831255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.832604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.842563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 480/978] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:34.843508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:34.925933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d7d60-7554-47b7-9ffc-cb8e3b67002c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.947394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cf64e-2012-469a-a62d-86bf9e630c99 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:34.960356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.962267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.962768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.963492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.982662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:34.983606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:34.983969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:34.984321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:34.985192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] RBAC: Adding query filter params (interface=admin, service_id=31df8cfe29af468bb3a12b5ddc800cf0, region_id=RegionOne) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:34.985594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:34.987064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:34.992991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 481/979] 10.222.0.83 () {56 vars in 1249 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/endpoints?service_id=31df8cfe29af468bb3a12b5ddc800cf0&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:34.998153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.031380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.032824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.094227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e343c2-b516-4f4d-8fa8-15030cb1f118 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.133668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 499/980] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 14:50:35.137018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.137907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.138436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.138866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.139150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.139546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.140921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e17b7-5ec5-40d7-8790-77ff63e749f5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.154429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 482/981] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:34 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:50:35.247687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf68e1-7c6c-451b-ae93-f38226e0edae None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.282215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.283138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.283492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.283832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.290541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6a2c-db62-4be3-8654-4cdf5006eaa7 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.290835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6a2c-db62-4be3-8654-4cdf5006eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.291108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6a2c-db62-4be3-8654-4cdf5006eaa7 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.292641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 483/982] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:35.295221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.296767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.297978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.298269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.298577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:35.315959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:35.317129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:35.318233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:35.318233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:35.318233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:35.318233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:35.318549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:35.318549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:35.319344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 500/983] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:35.325934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.392447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 484/984] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:35.414959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.429043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb11d6a-4f6b-4985-89d6-7ba73b0a9106 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.463675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.464344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.464623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.464898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.465608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:35.465911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.467306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.479565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 501/985] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:35.484775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.514089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4316eaf-4af7-47d4-8642-17f94aae28d1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.548679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.549349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.549629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.549901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.550648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.552018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.574442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 485/986] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:35 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:35.592075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2aed-644f-446f-84ca-8a322bfc4c6a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.635434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.636240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.636521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.636791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.644238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.645612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:35.661060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:35.662095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:35.663105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 502/987] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:35.669099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.771506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7437513-9e03-42ac-afe7-c1250f67326a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.808490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.809272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.809492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.809768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.810644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:35.810956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.812363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9f350-dd50-4641-be72-e81ef4279606 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.825312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 486/988] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:35.830570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:35.932350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07f16-1286-46d0-b30d-09975eecd12b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:35.964723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:35.965372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:35.965649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:35.965919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:35.966764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] RBAC: Adding query filter params (role.id=ca9dcb6ceb7740b8a98db73ff59974d9, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=7fc10eb4012647b9874edca4482f7c14) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:35.967072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:35.968473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01572025-8c91-4dd7-a503-e6642ae56527 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:35.974356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 503/989] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:35 2022] GET /identity/v3/role_assignments?user.id=7fc10eb4012647b9874edca4482f7c14&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=ca9dcb6ceb7740b8a98db73ff59974d9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:37.387001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455cc0c-5668-4358-8ced-20c4d19488e8 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.387529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455cc0c-5668-4358-8ced-20c4d19488e8 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.387529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455cc0c-5668-4358-8ced-20c4d19488e8 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.389078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 487/990] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:37.394041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.394526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.394646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.411102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:37.412264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:37.476662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 504/991] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:37.498515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455cc0c-5668-4358-8ced-20c4d19488e8 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:37.594320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455cc0c-5668-4358-8ced-20c4d19488e8 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:37.629274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:37.630102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.630382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.630683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.631344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:37.632796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:37.645537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:37.646893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 14:50:37.648051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:37.649162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 488/992] 10.222.0.83 () {56 vars in 1126 bytes} [Tue May 10 14:50:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:37.652832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:37.747605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053894ca-39b0-4346-8ef0-1a2178f847a3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:37.780279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:37.780949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.781234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.781511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.782213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:37.782550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:37.783894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4472f-067c-4b5c-a295-4e7a28e82100 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:37.793824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 505/993] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:37.795094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:37.802021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccabb5-1f58-4c37-94b2-0c02a7c3b6cf None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.802305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccabb5-1f58-4c37-94b2-0c02a7c3b6cf None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.802692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccabb5-1f58-4c37-94b2-0c02a7c3b6cf None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.804122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 506/994] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:37.809493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.809775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.810044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.825908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:37.827109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:37.894180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947683-51f6-47cf-9c10-604ea8a42a36 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:37.898498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 507/995] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:37.919495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:37.932851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:37.933524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:37.933805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:37.934086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:37.934813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:37.935119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:37.936499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:37.941286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 489/996] 10.222.0.83 () {56 vars in 1122 bytes} [Tue May 10 14:50:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:37.947096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.019539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e5c8-b333-42b3-8d51-ef9ddec8430f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.047124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa1cc1-1c3a-49ab-85fa-4e9d9aefad8c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.056672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.057488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.057769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.058037 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.058736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.060131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:38.072311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:38.073603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:38.074853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 508/997] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:38.079434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.087230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.088217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.088761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.089152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.090014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.091523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e843f-e3d8-4eff-addb-f8ec311451ee None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.106258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 490/998] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:38.197185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05682-e471-4e96-b093-f3b0481cc718 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.231984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d33b54-292b-4e5b-81b6-e95a444fe62b None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.232309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d33b54-292b-4e5b-81b6-e95a444fe62b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.232564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d33b54-292b-4e5b-81b6-e95a444fe62b None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.233954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 491/999] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:38.239053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.239617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.239741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.239914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.240008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.240184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.240279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.241181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] RBAC: Adding query filter params (name=compute) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:38.241513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.242927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.248408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 509/1000] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:38.254429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.258259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:38.259555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:38.328978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 492/1001] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:38.351329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.355055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016500c-f6e2-4d25-84ba-be3e09c8fcca None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.387791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.388456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.388738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.389014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.389711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] RBAC: Adding query filter params (type=compute) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:38.390016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.391403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.396299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 510/1002] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:38.402430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.449898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e480343-cbce-4a53-a468-9209e5a41940 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.486636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.487439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.487718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.487985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] PATH_INFO: `/v3/services/image` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.488633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.489992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:38.501790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:38.502857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:38.509435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:38.510206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5ffdd-bd8b-444a-9fca-e0b4033f493e None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.510284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 493/1003] 10.222.0.83 () {56 vars in 1105 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:38.510284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.536372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.537033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.537316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.537596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.538414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] RBAC: Adding query filter params (interface=public, service_id=b22e3379757c40799c18ef4f81f53b2a, region_id=RegionOne) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:38.538730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.540094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.549460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 511/1004] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/endpoints?service_id=b22e3379757c40799c18ef4f81f53b2a&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:38.550811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.608039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a97699-a212-49bc-bdb4-988d3c9ee25e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.641534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.642293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.642606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.642882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.643883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] RBAC: Adding query filter params (name=image) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:38.644130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.645599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.647283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ddfb7-b955-4f86-b0ae-82114e280fd5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.651476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 494/1005] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:38.656599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:38.681349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.682297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.682625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.682904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.683742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.685724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664c4db-1411-4670-bb0e-7d2e9b32ecff None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.692004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 512/1006] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:50:38.766344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d416a-069e-4b69-97a2-79624258a895 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:38.800903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:38.801584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:38.801866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:38.802139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:38.802974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] RBAC: Adding query filter params (type=image) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:38.803282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:38.805287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1133365-3a70-4b68-9eb5-fd35386c64ee None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:38.818706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 495/1007] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:40.341014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c22503-7343-430b-babc-6722a39a3430 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.341512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c22503-7343-430b-babc-6722a39a3430 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.341512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c22503-7343-430b-babc-6722a39a3430 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.343010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 513/1008] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:40.347998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.348387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.348546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.364600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:40.365984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:40.428727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 496/1009] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:40.451863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c22503-7343-430b-babc-6722a39a3430 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:40.547092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c22503-7343-430b-babc-6722a39a3430 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:40.578487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:40.579280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.579559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.579940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.581015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:40.582447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daba87-8195-48b1-904c-4d6d51ba2319 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:40.584215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 514/1010] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:40.590327 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:40.682267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef1db6-5c67-4275-847c-f4066204d3da None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:40.714360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:40.715026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.715305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.715578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.716278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:40.717679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b52cfb-7789-485c-8cb8-f9b04fcb6140 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:40.742284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 497/1011] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:40 2022] POST /identity/v3/projects => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:40.950090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147e9e-2dd9-49be-8802-4e199cc0184e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.950337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147e9e-2dd9-49be-8802-4e199cc0184e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.950607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147e9e-2dd9-49be-8802-4e199cc0184e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.951975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 515/1012] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:40.957217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:40.957502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:40.957775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:40.974450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:40.975625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:41.030355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febe1f9-8439-45af-957a-b1f0120d12df None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.030718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febe1f9-8439-45af-957a-b1f0120d12df None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.030961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febe1f9-8439-45af-957a-b1f0120d12df None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.032463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 516/1013] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:41.037355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.037719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.038072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.041237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 498/1014] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:41.059675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:41.060845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:41.065296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:41.126431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 517/1015] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:41.148731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:41.160988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569cc22-133b-43e0-8dfb-0bb833b23878 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:41.196032 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:41.196986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.197114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.197371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.198017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:41.199442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:41.211864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:41.212877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:41.214326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 499/1016] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:41.218776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:41.246788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2640e-3392-46f0-8e7a-ccfbfd15cc04 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:41.280658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:41.281326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.281612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.281887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.282638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:41.283994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:41.299305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 518/1017] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:50:41 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:41.317610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4f6a-1fdc-4595-8237-04c104cf89c0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:41.361153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:41.361840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.362096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.362372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.363174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:41.363483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:41.364902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:41.376004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 500/1018] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:41.377287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:41.483424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85fe65-a170-47aa-b190-9e8ea8b16c61 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:41.516318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:41.516977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.517257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.517525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.518213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:41.518563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:41.520001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de982ae-848a-4e2a-b33a-36b9048b4497 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:41.525221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 519/1019] 10.222.0.83 () {56 vars in 1112 bytes} [Tue May 10 14:50:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:41.531115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:41.628354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6d79-554b-4412-a317-b5791abef0f3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:41.660636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:41.661288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:41.661569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:41.661842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:41.662605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:41.664080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bade7a0-3e51-4ded-a917-ceeccaae2e2f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:41.690420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 501/1020] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:43.033558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf645ca-69b4-4258-ae80-a6def936009f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.034175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf645ca-69b4-4258-ae80-a6def936009f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.034175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf645ca-69b4-4258-ae80-a6def936009f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.035540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 520/1021] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:43.041226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.041627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.041795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.057846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:43.059033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:43.127833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 502/1022] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:43.152019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf645ca-69b4-4258-ae80-a6def936009f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:43.249693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf645ca-69b4-4258-ae80-a6def936009f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:43.281511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:43.282298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.282606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.282879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.284040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:43.285460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd66cd-3551-4734-955d-8d16e686ab1e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:43.287258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 521/1023] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:43.293237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:43.390094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c437be8-c5a1-49d4-b67d-179e8cf0402c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:43.426037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:43.426734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.427017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.427290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.428005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:43.430023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bc97c-1b10-4de6-a926-9df3e7b7a53e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:43.457164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 503/1024] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:50:43 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:50:43.562584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca86f9-9e7c-4366-83cc-5ac992834f1f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.562891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca86f9-9e7c-4366-83cc-5ac992834f1f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.563057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca86f9-9e7c-4366-83cc-5ac992834f1f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.564598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 522/1025] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:43.570775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.571065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.571341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.589395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:43.590629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:43.661205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 504/1026] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:43.684430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca86f9-9e7c-4366-83cc-5ac992834f1f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:43.787859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca86f9-9e7c-4366-83cc-5ac992834f1f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:43.825644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:43.826358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.826713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.827050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] PATH_INFO: `/v3/services/image` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.827708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:43.829108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d8846f95-6764-4101-a64f-11912ac69762 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:43.840776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:43.841790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:43.847707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 523/1027] 10.222.0.83 () {56 vars in 1105 bytes} [Tue May 10 14:50:43 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:43.849935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:43.952890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af69b0-9ae6-44a7-93fe-85629a5af21d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:43.982722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f004-a0f9-49de-ba65-8e876e426246 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.983031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f004-a0f9-49de-ba65-8e876e426246 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.983278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14f004-a0f9-49de-ba65-8e876e426246 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.984619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 524/1028] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:43.985382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:43.986037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.986320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.986623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.987319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] RBAC: Adding query filter params (name=image) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:43.987621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:43.988976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:43.989436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:43.989718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:43.989988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:43.994918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 505/1029] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:44.000284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.007083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:44.008762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:44.072791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 525/1030] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:44.095559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79873b11-a569-48d0-ab49-9a6d5495ea5a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.096043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.129180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.129835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.130114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.130383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.131101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] RBAC: Adding query filter params (type=image) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:44.131402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.132830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.137510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 506/1031] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:44.143229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.190453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b705ff-ebff-4c2e-89b8-755684196c24 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.222883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.223693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.223969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.224238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.235486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:44.241356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658334e-a1e6-4cbc-bf0e-c237ecf20916 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:44.244411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:44.245366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:44.251307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 526/1032] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:44.252027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.279906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.280563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.280842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.281112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.281887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] RBAC: Adding query filter params (interface=public, service_id=ff537fdc0e774cbb8424c4cf23898426, region_id=RegionOne) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:44.282192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.283651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.293638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 507/1033] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/endpoints?service_id=ff537fdc0e774cbb8424c4cf23898426&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:44.294651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.351866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465723b-f2d1-4207-b0e2-3fdb336c1bb1 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.383951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.384622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.384904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.385179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.385920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:44.386227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.387612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.390383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b9b21-5739-417c-b046-ac5678489447 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.395384 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 527/1034] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:44.401647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.425148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.425813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.426096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.426371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.427066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.428446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.434926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 508/1035] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/services => generated 1957 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:50:44.506300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b2b90-8655-41e9-bbbf-df7068717e2a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.542307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.543148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.543429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.543702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.555099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.556558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:44.574716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:44.575709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:44.576594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:44.576594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:44.576594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:44.576594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:44.576594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 528/1036] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:44.581829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.681704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb32487-c053-47c3-a9e9-1514f00a8d3a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.715429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.716094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.716371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.716642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.717349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:44.717650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.719031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.735426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 509/1037] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:44.736905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.830380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9f7e-ff81-4ae6-bf80-023ed17c10f3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:44.862209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:44.863105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:44.863453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:44.863789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:44.871558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:44.873038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:44.887493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:44.888644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:44.889831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:44.895292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 529/1038] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:44.896738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:44.996824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb37b-1af9-4b21-9fe7-5ab98ecd770a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.028610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.029238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.029520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.029790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.030631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:45.030934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.032340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.052884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 510/1039] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:45.054404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.149175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60909-aec0-40d7-a9ed-02d0fc7f4ed2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.180503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.181155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.181433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.181703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.245363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.246815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.342801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 530/1040] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:45.344203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.439882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e19f8-c802-4fe9-ba44-7ce557af572a None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.472589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.473439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.473717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.473980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.484150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.485571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:45.502294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:45.503310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:45.504280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:45.504280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:45.504280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:45.504280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:45.504280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 511/1041] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:45.509478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.608624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92aad1-6442-4d3a-a1da-3a0cc8ff3447 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.645415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.646105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.646409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.646685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.647421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:45.648286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.649676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5139418-37a8-4e52-82c1-f013f9098de4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.665757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 531/1042] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:45.667127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.761814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dbbbc-418a-4c3f-90bf-6ac32191bd09 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.793401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.794192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.794499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.794775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.801080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.802554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.811599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117574a-a905-4c83-9ac6-b235e8e71358 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.811893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117574a-a905-4c83-9ac6-b235e8e71358 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.812172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117574a-a905-4c83-9ac6-b235e8e71358 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.813698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 532/1043] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:45.816433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:45.817343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:45.818263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 512/1044] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:45.822104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.822411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.822708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.823692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.839211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:45.840381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:45.904233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 533/1045] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:45.927284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:45.930336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56082627-3f61-402c-a0c6-dd2834440ec2 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:45.964962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:45.965629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:45.965906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:45.966177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:45.966996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:45.967307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:45.968732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:45.980038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 513/1046] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:45.985639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.022872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238ea2-9c79-44ea-aba2-66f39e419150 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.055368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.056164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.056474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.056748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.066569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:46.074824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:46.075775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:46.081795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 534/1047] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:46.082906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.085193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0642-c55c-4736-b320-4976a842dc9f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.117253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.117903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.118184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.118482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.119304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] RBAC: Adding query filter params (role.id=ca9dcb6ceb7740b8a98db73ff59974d9, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:46.119612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.121030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.127477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 514/1048] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:45 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=ca9dcb6ceb7740b8a98db73ff59974d9 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:46.181713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176919f-19df-4456-a4aa-e6a907f420f5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.216578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.217254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.217534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.217807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.218590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:46.218902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.220272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.234018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 535/1049] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:46.235252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.346788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26d3cb-bb6b-4b22-a972-a0607071922b None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.379300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.380165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.380447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.380723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] PATH_INFO: `/v3/users/2c4f1561fb164df282434bf8446ae7c8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.393803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] RBAC: Authorizing `identity:get_user(user_id=2c4f1561fb164df282434bf8446ae7c8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.395271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907f0d-e859-4e33-a4da-6216db2b7089 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.397101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 515/1050] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/users/2c4f1561fb164df282434bf8446ae7c8 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:46.402648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.499788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ce-ae4e-4919-9f09-bcc3c9775700 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.532940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.533588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.533871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.534147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.597612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.599091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.675844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cacbd3-4d72-4408-8b2b-e310dbf86f12 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.676168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cacbd3-4d72-4408-8b2b-e310dbf86f12 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.676393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cacbd3-4d72-4408-8b2b-e310dbf86f12 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.677883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 516/1051] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:46.682747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.683202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.683323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.698038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 536/1052] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:46.699136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.704263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:46.705513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:46.769288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 517/1053] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:46.791530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9433be2-de97-4bf9-bbf0-84bd9d417a89 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.792486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.828197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.829802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.830081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.830352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.833822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.836050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.843740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 537/1054] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:46.844965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.896155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e2a6a-8ba9-43c7-8072-cad0b6f15fbc None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.932481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.933263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.933540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.933809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] PATH_INFO: `/v3/services/image` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.934471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.935822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.942307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9195ab7-c20f-4230-aa2c-f0c0312b8561 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:46.947167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:46.948178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:46.954099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 518/1055] 10.222.0.83 () {56 vars in 1105 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:46.954971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:46.976572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:46.977229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:46.977511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:46.977786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:46.978632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950, user.id=2c4f1561fb164df282434bf8446ae7c8) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:46.979090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:46.980463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:46.985747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 538/1056] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/role_assignments?user.id=2c4f1561fb164df282434bf8446ae7c8&scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:47.056066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48d7e8-e771-4c57-a1c2-d3de4f610acb None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:47.090483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:47.091112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:47.091393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:47.091663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:47.092373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] RBAC: Adding query filter params (name=image) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:47.092674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:47.094049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:47.106227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 519/1057] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:47.106448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:47.215383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198355b-5480-4d0b-9f30-d6790fc85499 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:47.252252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:47.253222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:47.253714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:47.254175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:47.255107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] RBAC: Adding query filter params (type=image) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:47.255591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:47.257142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1168ba-3cf4-4b79-b9e7-a2b4f384cb9a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:47.262556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 539/1058] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:47.268099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:47.363313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcef739-6ef5-412b-ba03-dadcc0a56f54 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:47.395909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:47.396557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:47.396834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:47.397102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:47.398382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:47.399762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e94c0-11c6-4669-a068-f3ff17f1d2ca None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:47.419771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 520/1059] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:50:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:48.345522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b3b3-932e-429c-9713-f663a0db9564 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:48.346033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b3b3-932e-429c-9713-f663a0db9564 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:48.346033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b3b3-932e-429c-9713-f663a0db9564 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:48.347428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 540/1060] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:48.352494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:48.352787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:48.353066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:48.369907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:48.371092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:48.439141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 521/1061] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:48.461903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638b3b3-932e-429c-9713-f663a0db9564 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:48.565045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638b3b3-932e-429c-9713-f663a0db9564 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:48.598267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:48.599098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:48.599374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:48.599641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:48.609889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:48.618784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:48.619755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:48.620622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 541/1062] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:48.626033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:48.718794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58c821-2c92-4ebd-b4a8-e4fb440ee702 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:48.751654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:48.752320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:48.752602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:48.752874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:48.753611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:48.753913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:48.755293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:48.763161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 522/1063] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:48.768747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:48.873731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984da521-2d5e-405a-856b-556b370c8e2f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:48.904590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:48.905371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:48.908448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:48.908448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:48.921465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:48.923038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:48.939617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:48.940491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:48.941470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:48.941470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:48.941470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:48.941470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:48.942695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 542/1064] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:48.946356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.043978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f6c3-8f9c-4cb7-8760-54ef95558072 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.076593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.077261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.077541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.077816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.078554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:49.078860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.080308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f623240-3836-4f9d-b213-2442f5cacb90 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.092416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 523/1065] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:49.097755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.200473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2ed9-8e90-4430-a8c2-d3fb4f9ed985 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.209209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247bcc-c1c0-42df-bc60-552ba93bcc09 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.209500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247bcc-c1c0-42df-bc60-552ba93bcc09 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.209779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247bcc-c1c0-42df-bc60-552ba93bcc09 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.211824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 524/1066] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:50:49.216557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.216844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.217121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.234535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:49.235701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:49.237391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.238189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.238512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.238792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.246712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.248144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:49.263048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:49.264005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:49.265003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 543/1067] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:49.269739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.305466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 525/1068] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:49.327654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.362783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df55a-a4be-45ee-b92f-64d7a0b27d8b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.395429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.396095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.396372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.396641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.397521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:49.398333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.399768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.416020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 544/1069] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:49.421429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.422222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab9b48-4afc-4141-8286-37f4eaa6fc13 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.456141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.456867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.457146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.457417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.468994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:49.476555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:49.477568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:49.479039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 526/1070] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:49.484372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.524347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384b50-fba2-4f61-aab5-1d3ea598f218 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.559661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.560454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.560733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.561054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.571175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.572734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.583914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b419c-e6a7-4c56-af21-a0c2490b4f51 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:49.590175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:49.591137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:49.592073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:49.592073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:49.592073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:49.592073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:49.592073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 545/1071] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:49.596864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.620832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.621547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.621837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.622114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.622884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:49.623198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.624562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.637766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 527/1072] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:49.638687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.695475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042ec15-b21a-41ec-88a9-2bcb7833368f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.726689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.727277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.727557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.727831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.728542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:49.728849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.730194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee16f5-f0b8-48bb-9d0e-4da6835d53bb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.731440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fe16d-26b7-4f75-800b-b02dc9b0cbd0 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.741074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 546/1073] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:49.741074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07b02-effe-459d-abed-7d331f8aec47 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.741074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07b02-effe-459d-abed-7d331f8aec47 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.741074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07b02-effe-459d-abed-7d331f8aec47 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.741074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 547/1074] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:49.747124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab07b02-effe-459d-abed-7d331f8aec47 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.763932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.764795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.765072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.765340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] PATH_INFO: `/v3/users/2c4f1561fb164df282434bf8446ae7c8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.766518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] RBAC: Authorizing `identity:get_user(user_id=2c4f1561fb164df282434bf8446ae7c8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.767932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048219a-1e13-43c9-a7ec-a4930d879e0a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.769516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 528/1075] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/users/2c4f1561fb164df282434bf8446ae7c8 => generated 285 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:49.770709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.771189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.771299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.786812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:49.787976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:49.842145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab07b02-effe-459d-abed-7d331f8aec47 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.848548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 529/1076] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:49.849352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.874728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.875516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.875800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.876074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.883008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.884426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:49.898911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:49.899879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 548/1077] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:49.900817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.948817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89148e32-936a-4aa7-9dd4-a580b16032f6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:49.979866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:49.980658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:49.980935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:49.981209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:49.982355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:49.983788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:49.985441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 530/1078] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:49.986360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:49.992817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712d4d8-8fa6-4a51-afa6-eb90b589bc81 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.024528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.025160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.025440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.025711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.078097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddead7-53b0-42ba-ba3d-34a0cb37601e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.086894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.088313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.109791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.110468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.110756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.111035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.111802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:50.112114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.113500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.125284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 531/1079] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:50.125883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:50.179608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 549/1080] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:50.180404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:50.217808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e023-0fec-441f-870b-d10848ef6aed None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.249676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.250521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.250803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.251085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950/users/2c4f1561fb164df282434bf8446ae7c8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.253046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ebb0084e0894c1aa8ddfdc384d94950, user_id=2c4f1561fb164df282434bf8446ae7c8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.254512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.268042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 532/1081] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:50 2022] PUT /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950/users/2c4f1561fb164df282434bf8446ae7c8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:50.268821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:50.274500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f372f8-3f04-4b73-9795-7c430a89c326 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.311790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.312699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.312976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.313251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/ca9dcb6ceb7740b8a98db73ff59974d9` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.315554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=5bb3df7bdd8847a1a3e17993b1f466a8, role_id=ca9dcb6ceb7740b8a98db73ff59974d9)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.317008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.331225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 550/1082] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:50:50 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/5bb3df7bdd8847a1a3e17993b1f466a8/roles/ca9dcb6ceb7740b8a98db73ff59974d9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:50:50.382735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31d20e-e668-4cff-be63-173cfc532956 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.421552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.422811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.423789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.424079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.436331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.437822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-9a8f469d-f77d-4750-a432-8429a0685a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:50.456587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:50.457586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:50.458987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:50.458987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:50.458987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:50.458987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:50.458987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 533/1083] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:50.464603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:50.574591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc703303-c0f8-4c6a-bec3-f23d36fa9f62 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:50.608053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:50.608719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:50.609003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:50.609276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:50.609988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:50.610291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:50.611680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad3d68-e160-45b9-8678-540fb59b53aa None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:50.623242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 551/1084] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:52.515847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68943c6-5eea-45b5-8d13-9af97a587c4f None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.516388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68943c6-5eea-45b5-8d13-9af97a587c4f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.516388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68943c6-5eea-45b5-8d13-9af97a587c4f None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.517928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 534/1085] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:52.522990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.523378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.523652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.540098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:52.541243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:52.602931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 552/1086] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:52.626517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:52.627058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238d3c-7180-47f3-9616-809bf16fa32d None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.627349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238d3c-7180-47f3-9616-809bf16fa32d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.627624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238d3c-7180-47f3-9616-809bf16fa32d None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.629118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 535/1087] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:52.634048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.634334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.634635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.651565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:52.652730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:52.715884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 536/1088] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:50:52.719547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bd575-1e10-4128-8847-3cd824ffd799 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:52.739315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:52.751511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:52.752302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.752584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.752859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.762289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:52.769938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:52.771055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:50:52.771982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 553/1089] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:52.777360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:52.828802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3ef08-f47b-4069-9b67-6d9f222c5a25 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:52.860438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:52.861224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.861502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.861773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.868135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046b26-5042-4954-8aa7-2a2700cd1302 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:52.871351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:50:52.879435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:52.880597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:52.881655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 537/1090] 10.222.0.83 () {56 vars in 1120 bytes} [Tue May 10 14:50:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:52.886507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:52.902420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:52.903073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:52.903353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:52.903625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:52.904351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:52.904660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:52.906014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:52.914349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 554/1091] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:52.919896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:52.986113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635b0b-0c07-47c1-9090-446b9b192daf None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.011114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495cd82-4994-4fb7-953c-bd8aa04df87a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.017246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.017899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.018173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.018475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.019203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:53.019514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.020912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.028812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 538/1092] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:50:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:53.034350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.042904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.043773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.044060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.044332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] PATH_INFO: `/v3/users/2c4f1561fb164df282434bf8446ae7c8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.045537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] RBAC: Authorizing `identity:get_user(user_id=2c4f1561fb164df282434bf8446ae7c8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.046992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.049420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 555/1093] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:50:52 2022] GET /identity/v3/users/2c4f1561fb164df282434bf8446ae7c8 => generated 285 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:53.054058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.125715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a57d-a490-44c5-bd56-0d129cb28a59 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.148422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c1e36-80c2-4b31-ae4a-a2c55e9061e8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.157428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.158287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.158688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.158963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.169413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.170888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.179464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.180092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.180382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.180653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:53.187316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:53.188256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:53.189240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:53.189240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:53.189240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:53.189240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:53.189240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 539/1094] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:53.194063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.244013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.245454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.285069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77befbfe-27ac-4e7c-9671-a99299c9f246 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.315362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.315995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.316268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.316538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.317236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:53.317533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.318894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.330947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 540/1095] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:53.340503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 556/1096] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 14:50:53.340653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.341622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.428123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af22c-f493-46ef-a4a8-291112a0ecc5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.434478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5e847-7d02-47d7-9c7e-69ea780df1c5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.460071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.460845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.461138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.461415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.466587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.467437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.467718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.467989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.468097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.469150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.469465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.470637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.472334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 557/1097] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:53.478820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:50:53.484008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:53.484998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:53.485995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 541/1098] 10.222.0.83 () {56 vars in 1093 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:50:53.491553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.572584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566ec05-a492-426d-b82a-55cb0f74ac7d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.583358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7a7f-ce03-4f81-82dd-19ee1f996563 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.603663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.604328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.604612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.604943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.605765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950, user.id=2c4f1561fb164df282434bf8446ae7c8) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:53.606090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.607542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.613121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 558/1099] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/role_assignments?user.id=2c4f1561fb164df282434bf8446ae7c8&scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:53.618828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.619509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.619788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.620061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.620902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:53.622293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.624391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.647085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 542/1100] 10.222.0.83 () {56 vars in 1133 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:53.653059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:53.756277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a4ae-568e-4955-b6fc-eea7242a7639 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:53.791619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:53.792273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:53.792558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:53.792842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:53.857759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:53.859239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:53.956683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 559/1101] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:50:53.964017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:54.060264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595a5d-746f-4de8-af36-27fbecbac55d None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:54.092152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:54.092935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:54.093215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:54.093488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:54.103641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:54.105073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:50:54.123580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:54.124549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:54.125466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:50:54.125466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:50:54.125466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:50:54.125466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:54.125466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 543/1102] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:54.130452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:54.222823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda963-ae34-49ed-9d74-781b937a3cec None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:54.255460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:54.256166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:54.256451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:54.256726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:54.258034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:54.258342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:54.259719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:54.270789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 560/1103] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:54.276425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:54.369876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7856-dc11-42c5-8b18-0e6f6a51ff9f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:54.406421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:54.407147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:54.407429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:54.407703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:54.414121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:54.415587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:50:54.429406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:50:54.430360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:50:54.431306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 544/1104] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:50:54.436544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:54.530315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285faec-7188-43ca-b8be-a91029fcf85b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:54.562178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:54.562995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:54.563281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:54.563556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:54.564328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:54.564637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:54.566042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4fc58-de65-4632-8d36-1ff719188f00 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:54.577983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 561/1105] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:50:54.584046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:54.676394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94465f8-4e0a-4a32-a7ee-97c8719d11b5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:54.708487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:54.709138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:54.709419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:54.709693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:54.710550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] RBAC: Adding query filter params (role.id=ca9dcb6ceb7740b8a98db73ff59974d9, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=5bb3df7bdd8847a1a3e17993b1f466a8) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:54.710863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:54.712278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1f957-8f31-4ec2-8df9-771d750e6491 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:54.717886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 545/1106] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:54 2022] GET /identity/v3/role_assignments?user.id=5bb3df7bdd8847a1a3e17993b1f466a8&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=ca9dcb6ceb7740b8a98db73ff59974d9 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:55.872490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbb6d-7182-4654-bd14-81b9655dad2e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:55.873139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbb6d-7182-4654-bd14-81b9655dad2e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:55.873139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbb6d-7182-4654-bd14-81b9655dad2e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:55.874376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 562/1107] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:55.879552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:55.880008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:55.880140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:55.896296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:55.897962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:55.958669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 546/1108] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:55.983374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cbb6d-7182-4654-bd14-81b9655dad2e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:56.075329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cbb6d-7182-4654-bd14-81b9655dad2e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:56.106035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:56.106824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:56.107100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:56.107369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:56.108501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:56.109903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a067f-8f15-45a1-ba0f-eb3de1725b2e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:56.111480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 563/1109] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:50:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:56.117213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:56.207404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84851dd6-ef5c-4c45-9007-3c73b00940ef None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:56.238303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:56.238966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:56.239244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:56.239514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:56.240204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:56.241906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ddd6e-6c29-4c9c-ae57-9ef898b5cccb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:56.261372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 547/1110] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:50:56 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:50:58.494097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad24-b0ad-4a1e-ad3f-a54d99f8a8d4 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:58.494774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad24-b0ad-4a1e-ad3f-a54d99f8a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:58.494774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad24-b0ad-4a1e-ad3f-a54d99f8a8d4 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:58.496040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 564/1111] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:50:58.501175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:58.501612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:58.501714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:58.517761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:50:58.518966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:58.580591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 548/1112] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:50:58.603894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ad24-b0ad-4a1e-ad3f-a54d99f8a8d4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:58.697114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ad24-b0ad-4a1e-ad3f-a54d99f8a8d4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:58.728503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:58.729327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:58.729607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:58.729880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] PATH_INFO: `/v3/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:58.731345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] RBAC: Authorizing `identity:get_role(role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:58.732682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:58.734002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 565/1113] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:50:58 2022] GET /identity/v3/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:58.739530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:58.832025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d26b-739f-4fb0-b3c6-2e9b8f49b684 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:58.863564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:58.864420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:58.864700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:58.864975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] PATH_INFO: `/v3/users/f72beab6b01a4beea783122c4839f9bf` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:58.877873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] RBAC: Authorizing `identity:get_user(user_id=f72beab6b01a4beea783122c4839f9bf)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:58.879334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:58.880949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 549/1114] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:50:58 2022] GET /identity/v3/users/f72beab6b01a4beea783122c4839f9bf => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:58.887209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:58.978162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e8c-5ac3-45c5-938c-2fd55f461c2a None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:59.010005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:59.010662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:59.010943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:59.011218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:59.071898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:59.073298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:59.161943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 566/1115] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:50:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 14:50:59.168206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:59.259758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf6494-a960-4f76-8cfa-d105ea892f73 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:59.290936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:59.291764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:59.292042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:59.292312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:59.293461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:59.294905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a31e28-15cc-4d6f-8b42-de83bf966f44 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:59.296537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 550/1116] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:50:59 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:50:59.302291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:59.395608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0dcae-90d6-4d17-a536-5f7c79a208c4 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:59.427753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:59.428335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:59.428641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:59.428915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:59.429732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950, user.id=f72beab6b01a4beea783122c4839f9bf) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:50:59.430039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:59.431477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a91e4-0c86-4eaa-8b2a-ea12b14d575d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:59.436574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 567/1117] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:50:59 2022] GET /identity/v3/role_assignments?user.id=f72beab6b01a4beea783122c4839f9bf&scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:01.617232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468e5e-0c17-4326-8735-7787c46ffc51 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:01.617772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468e5e-0c17-4326-8735-7787c46ffc51 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:01.617772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468e5e-0c17-4326-8735-7787c46ffc51 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:01.619155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 551/1118] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:01.624334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:01.624741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:01.624879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:01.641990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:01.643212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:01.703953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 568/1119] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:51:01.725577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468e5e-0c17-4326-8735-7787c46ffc51 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:01.819727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468e5e-0c17-4326-8735-7787c46ffc51 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:01.850029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:01.850887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:01.851164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:01.851431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] PATH_INFO: `/v3/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:01.852863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] RBAC: Authorizing `identity:get_role(role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:01.854209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:01.855575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 552/1120] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:01 2022] GET /identity/v3/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 223 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:01.861166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:01.952620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676c0a2-7079-4d26-843f-396036b2f362 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:01.983152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:01.984009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:01.984284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:01.984552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] PATH_INFO: `/v3/users/f72beab6b01a4beea783122c4839f9bf` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:01.985702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] RBAC: Authorizing `identity:get_user(user_id=f72beab6b01a4beea783122c4839f9bf)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:01.987140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:01.988744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 569/1121] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:01 2022] GET /identity/v3/users/f72beab6b01a4beea783122c4839f9bf => generated 286 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:01.995151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:02.086150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc12b-2aa9-4abe-a782-32b51f6813f4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:02.117887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:02.118747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:02.119024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:02.119295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:02.120437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:02.121843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5467ae-d5f7-42cf-8f84-08aea9c29f5d None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:02.123491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 553/1122] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:01 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:02.129156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:02.219846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51e49-2291-42c4-ab8f-40fbf75f5362 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:02.251352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:02.252213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:02.252491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:02.252760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950/users/f72beab6b01a4beea783122c4839f9bf/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:02.254911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ebb0084e0894c1aa8ddfdc384d94950, user_id=f72beab6b01a4beea783122c4839f9bf, role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:02.256295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded86666-9997-4426-b8a4-21773cef6149 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:02.268448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 570/1123] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:51:02 2022] PUT /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950/users/f72beab6b01a4beea783122c4839f9bf/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:51:04.454080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cc57-bb00-4464-bf4c-2b01de885d0a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:04.454674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cc57-bb00-4464-bf4c-2b01de885d0a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:04.454674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cc57-bb00-4464-bf4c-2b01de885d0a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:04.456020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 554/1124] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:04.461730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:04.462153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:04.462261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:04.478125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:04.479303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:04.541151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 571/1125] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:51:04.563701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cc57-bb00-4464-bf4c-2b01de885d0a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:04.660671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cc57-bb00-4464-bf4c-2b01de885d0a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:04.692497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:04.693339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:04.693622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:04.693899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] PATH_INFO: `/v3/roles/6fdd47a804ed45e78d3a53207cd8d795` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:04.695942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] RBAC: Authorizing `identity:get_role(role_id=6fdd47a804ed45e78d3a53207cd8d795)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:04.697301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:04.698693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 555/1126] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:04 2022] GET /identity/v3/roles/6fdd47a804ed45e78d3a53207cd8d795 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:04.704610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:04.794568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfd49a-2719-4929-bee9-5a429a8c42fc None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:04.825230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:04.826077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:04.826354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:04.826728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] PATH_INFO: `/v3/users/f72beab6b01a4beea783122c4839f9bf` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:04.827886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=f72beab6b01a4beea783122c4839f9bf)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:04.829285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:04.830885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 572/1127] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:04 2022] GET /identity/v3/users/f72beab6b01a4beea783122c4839f9bf => generated 286 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:04.836989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:04.929714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-dc7a-465d-aac1-d5be854efe63 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:04.961282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:04.961861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:04.962145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:04.962443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:05.025679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:05.027131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:05.116610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 556/1128] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:05.122474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:05.214024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbfd5e-1b16-4c54-9de2-fe8ceafcb6c1 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:05.244674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:05.245513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:05.245795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:05.246066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] PATH_INFO: `/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:05.247249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=4ebb0084e0894c1aa8ddfdc384d94950)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:05.248648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b7a6f-4d94-4cd6-b33a-40e29a66c8e8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:05.250288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 573/1129] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:05 2022] GET /identity/v3/projects/4ebb0084e0894c1aa8ddfdc384d94950 => generated 311 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:05.256094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:05.348067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656de9b-c9e8-44fe-91ac-3bbee51034ac None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:05.379126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:05.379761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:05.380043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:05.380318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:05.381125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] RBAC: Adding query filter params (role.id=6fdd47a804ed45e78d3a53207cd8d795, scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950, user.id=f72beab6b01a4beea783122c4839f9bf) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:05.381953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:05.383395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa29a40-e6bc-445f-81b4-ac8811276cf5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:05.388883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 557/1130] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:51:05 2022] GET /identity/v3/role_assignments?user.id=f72beab6b01a4beea783122c4839f9bf&scope.project.id=4ebb0084e0894c1aa8ddfdc384d94950&role.id=6fdd47a804ed45e78d3a53207cd8d795 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:07.587047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021340-6320-469b-a117-83fe8c990301 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:07.587600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021340-6320-469b-a117-83fe8c990301 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:07.587600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021340-6320-469b-a117-83fe8c990301 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:07.588956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 574/1131] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:07.594192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:07.594615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:07.594777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:07.612355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:07.613529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:07.676019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 558/1132] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:07.698474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021340-6320-469b-a117-83fe8c990301 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:07.796596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021340-6320-469b-a117-83fe8c990301 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:07.827160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:07.827945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:07.828223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:07.828493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:07.829642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:07.831109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54398c-3220-4f1e-a7c2-7a03a48d1ee8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:07.832707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 575/1133] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:07.839004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:07.930642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25250f3-2080-4b8c-8fea-daa700bfb2c5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:07.962095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:07.962771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:07.963057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:07.963332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:07.964041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:07.965460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161c3b4-c430-4ed9-bbe9-85a3b3240b4a None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:07.989381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 559/1134] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:51:07 2022] POST /identity/v3/projects => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:51:10.215369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3bd8-83be-4f57-b64f-2043f2082059 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:10.215939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3bd8-83be-4f57-b64f-2043f2082059 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:10.215939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3bd8-83be-4f57-b64f-2043f2082059 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:10.217198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 576/1135] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:10.222346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:10.222744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:10.222926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:10.239459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:10.240611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:10.301380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 560/1136] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:10.325755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a3bd8-83be-4f57-b64f-2043f2082059 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:10.423051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a3bd8-83be-4f57-b64f-2043f2082059 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:10.454030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:10.454822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:10.455103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:10.455377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] PATH_INFO: `/v3/domains/default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:10.456524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:10.457945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759caa1-65d6-468e-9413-b6655814a97f None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:10.460099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 577/1137] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:51:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:10.465965 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:10.566580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a8c2-ac76-4dcf-8a1d-79730c169748 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:10.599241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:10.599893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:10.600199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:10.600473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:10.601164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:10.602601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0bec-250e-4180-b89d-fa9398ff9f45 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:10.621092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 561/1138] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:51:10 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:51:12.846383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bb38-f644-45d4-b441-664997b93c7d None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:12.846933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bb38-f644-45d4-b441-664997b93c7d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:12.847056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bb38-f644-45d4-b441-664997b93c7d None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:12.848331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 578/1139] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:12.853660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:12.854073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:12.854214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:12.870454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:12.871613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:12.933107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 562/1140] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:12.955766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bb38-f644-45d4-b441-664997b93c7d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.048601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bb38-f644-45d4-b441-664997b93c7d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.079854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.080623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.080904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.081177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.091186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:13.098748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:13.099923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:13.101805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 579/1141] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:13.106233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.196593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d7b15-abc6-48ac-b330-91445ea9a4bf None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.228222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.228862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.229142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.229415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.230139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:13.230466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:13.231822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:13.239280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 563/1142] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:51:13.244985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.336908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888df5d-44b9-426b-b1d5-cbb71c95659f None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.367343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.368124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.368399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.368666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] PATH_INFO: `/v3/users/c16bafea5aee4858b7b839cccdf4e62f` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.380785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] RBAC: Authorizing `identity:get_user(user_id=c16bafea5aee4858b7b839cccdf4e62f)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:13.382746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:13.384331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 580/1143] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:13 2022] GET /identity/v3/users/c16bafea5aee4858b7b839cccdf4e62f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:13.389786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.483467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579a2d4-c746-483e-910f-eb3bf4b76b34 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.514450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.515076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.515358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.515632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.576969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:13.578413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:13.669483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 564/1144] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:13.675241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.774683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c1290-50b0-456a-8c0b-c41dc84035cb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.806823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.807658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.807937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.808211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] PATH_INFO: `/v3/projects/046525dc6e1241d59f9147bb42ffdbf4` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.809360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=046525dc6e1241d59f9147bb42ffdbf4)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:13.810799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e64fff-5103-4074-a1bc-863607fb4ca3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:13.812476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 581/1145] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:13 2022] GET /identity/v3/projects/046525dc6e1241d59f9147bb42ffdbf4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:13.818538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:13.909209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720cc71-1b27-4801-b0da-5a844c68b279 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:13.940929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:13.941509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:13.941788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:13.942063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:13.942908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=046525dc6e1241d59f9147bb42ffdbf4, user.id=c16bafea5aee4858b7b839cccdf4e62f) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:13.943218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:13.945202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96be6a-815b-407f-98f0-9176f39b7230 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:13.950298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 565/1146] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:51:13 2022] GET /identity/v3/role_assignments?user.id=c16bafea5aee4858b7b839cccdf4e62f&scope.project.id=046525dc6e1241d59f9147bb42ffdbf4&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:16.111583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d21-163f-4c36-b789-dceafb822da6 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.112107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d21-163f-4c36-b789-dceafb822da6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.112227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d21-163f-4c36-b789-dceafb822da6 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.113499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 582/1147] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:16.118945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709181-7b46-4d15-886b-4f280571be5f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.119322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709181-7b46-4d15-886b-4f280571be5f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.119490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709181-7b46-4d15-886b-4f280571be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.136005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-52709181-7b46-4d15-886b-4f280571be5f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:16.137156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-52709181-7b46-4d15-886b-4f280571be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:16.198125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 566/1148] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:16.220152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185d21-163f-4c36-b789-dceafb822da6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:16.313947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185d21-163f-4c36-b789-dceafb822da6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:16.346093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:16.346887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.347171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.347445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.356757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:16.364340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:16.365426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:16.366830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 583/1149] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:16.371531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709181-7b46-4d15-886b-4f280571be5f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:16.468455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709181-7b46-4d15-886b-4f280571be5f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:16.499426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:16.500074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.500358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.500630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.501354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:16.501657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:16.503047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:16.511182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 567/1150] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:16.516998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:16.614567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343214d3-3f24-4d5b-95e0-f00d3bd5c428 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:16.645851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:16.646780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.647587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.647859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] PATH_INFO: `/v3/users/c16bafea5aee4858b7b839cccdf4e62f` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.649034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] RBAC: Authorizing `identity:get_user(user_id=c16bafea5aee4858b7b839cccdf4e62f)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:16.650481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:16.652096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 584/1151] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:16 2022] GET /identity/v3/users/c16bafea5aee4858b7b839cccdf4e62f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:16.657734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:16.749735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead3bc4-995e-43df-ab0c-5c90862d16c5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:16.780438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:16.781278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.781558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.781830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] PATH_INFO: `/v3/projects/046525dc6e1241d59f9147bb42ffdbf4` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.783004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=046525dc6e1241d59f9147bb42ffdbf4)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:16.784422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93378f-4809-468d-b6e8-d7a37e48a5d9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:16.786065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 568/1152] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:16 2022] GET /identity/v3/projects/046525dc6e1241d59f9147bb42ffdbf4 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:16.792042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:16.883629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cf3c7-54fc-4851-a228-12ce5a276d7d None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:16.914000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:16.914896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:16.915174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:16.915442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] PATH_INFO: `/v3/projects/046525dc6e1241d59f9147bb42ffdbf4/users/c16bafea5aee4858b7b839cccdf4e62f/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:16.917361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] RBAC: Authorizing `identity:create_grant(project_id=046525dc6e1241d59f9147bb42ffdbf4, user_id=c16bafea5aee4858b7b839cccdf4e62f, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:16.918809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410e0d-9c10-48d7-a977-5c55ed5c2406 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:16.931819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 585/1153] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:51:16 2022] PUT /identity/v3/projects/046525dc6e1241d59f9147bb42ffdbf4/users/c16bafea5aee4858b7b839cccdf4e62f/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:51:19.104940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478864-6025-4b3c-a306-e7def10245da None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.105464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478864-6025-4b3c-a306-e7def10245da None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.105464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52478864-6025-4b3c-a306-e7def10245da None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.106873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 569/1154] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:19.111943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.112379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.112502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.128697 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:19.129855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:19.190569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 586/1155] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:51:19.213077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52478864-6025-4b3c-a306-e7def10245da None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:19.306169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52478864-6025-4b3c-a306-e7def10245da None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:19.337380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:19.338145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.338445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.338716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.347979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:19.355561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:19.356635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:51:19.357664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 570/1156] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:19.362278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:19.452281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee463e3a-805b-4f06-9f8c-937a59a9b5ab None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:19.484548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:19.485208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.485489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.485763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.486528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:19.486834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:19.488194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:19.496310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 587/1157] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:19.502075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:19.594135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80975d90-2907-49a7-aa8d-79bd30a50061 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:19.624878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:19.625720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.625999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.626271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] PATH_INFO: `/v3/users/c16bafea5aee4858b7b839cccdf4e62f` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.627450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] RBAC: Authorizing `identity:get_user(user_id=c16bafea5aee4858b7b839cccdf4e62f)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:19.628867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:19.630452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 571/1158] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:19 2022] GET /identity/v3/users/c16bafea5aee4858b7b839cccdf4e62f => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:19.636386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:19.728502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc9279-793c-4735-ac98-ae5cd0315ea2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:19.761450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:19.762081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:19.762361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:19.762664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:19.827062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:19.828490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:19.918443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 588/1159] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:19.924187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:20.018042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adf3bf-3fd5-4410-ba04-a45710015506 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:20.049181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:20.050019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:20.050298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:20.050606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] PATH_INFO: `/v3/projects/046525dc6e1241d59f9147bb42ffdbf4` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:20.051755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] RBAC: Authorizing `identity:get_project(project_id=046525dc6e1241d59f9147bb42ffdbf4)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:20.053735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8d0b6-5620-49f4-8b41-2fec14e6f386 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:20.055413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 572/1160] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:19 2022] GET /identity/v3/projects/046525dc6e1241d59f9147bb42ffdbf4 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:20.061178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:20.161406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f54fc5-4a34-4479-bc20-b0d586c7b5e2 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:20.193104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:20.193759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:20.194036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:20.194306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:20.195139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=046525dc6e1241d59f9147bb42ffdbf4, user.id=c16bafea5aee4858b7b839cccdf4e62f) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:20.195443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:20.196846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f153-7648-440b-87d3-555f49327b95 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:20.202257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 589/1161] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:51:20 2022] GET /identity/v3/role_assignments?user.id=c16bafea5aee4858b7b839cccdf4e62f&scope.project.id=046525dc6e1241d59f9147bb42ffdbf4&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:22.404756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f0e1-24be-421d-9cf5-267dc4e78763 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:22.405318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f0e1-24be-421d-9cf5-267dc4e78763 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:22.405318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f0e1-24be-421d-9cf5-267dc4e78763 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:22.406701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 573/1162] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:22.411995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:22.412911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:22.413126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:22.429659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:22.430860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:22.492431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 590/1163] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:22.514079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03f0e1-24be-421d-9cf5-267dc4e78763 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:22.607369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03f0e1-24be-421d-9cf5-267dc4e78763 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:22.638641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:22.639260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:22.639539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:22.639809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:22.709956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:22.711354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:22.801282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 574/1164] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:22.807281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:22.900811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e882f-806b-4948-bb6a-da1fb940b62c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:22.932532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:22.933334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:22.933612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:22.933887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:22.940191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:22.941612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-66e5445e-0386-4ef0-9f33-d069375bda04 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:22.955185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:22.956379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:22.957228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 591/1165] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:51:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:22.962933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:23.053670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc175032-71c0-4ae6-9405-0ef56892bca1 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:23.085374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:23.086007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:23.086289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:23.086597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:23.087302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:23.087606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:23.088948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2664d-642e-495b-b6d8-b13396081d70 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:23.095680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 575/1166] 10.222.0.83 () {56 vars in 1115 bytes} [Tue May 10 14:51:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:25.277905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e1191-3ca2-43f6-a2fb-ef0382a85b74 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:25.278542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e1191-3ca2-43f6-a2fb-ef0382a85b74 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:25.278542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e1191-3ca2-43f6-a2fb-ef0382a85b74 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:25.279740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 592/1167] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:25.284985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351500d3-5669-420c-8d03-4f76f4944370 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:25.285388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351500d3-5669-420c-8d03-4f76f4944370 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:25.285532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351500d3-5669-420c-8d03-4f76f4944370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:25.301863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-351500d3-5669-420c-8d03-4f76f4944370 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:25.303639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-351500d3-5669-420c-8d03-4f76f4944370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:25.363172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 576/1168] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:51:25.385371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e1191-3ca2-43f6-a2fb-ef0382a85b74 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:25.479298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e1191-3ca2-43f6-a2fb-ef0382a85b74 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:25.510507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:25.511140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:25.511421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:25.511699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:25.512431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:25.513765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073ec6b-4ab0-489a-8ebd-ee30eb8cdc9d None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:25.528302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 593/1169] 10.222.0.83 () {60 vars in 1138 bytes} [Tue May 10 14:51:25 2022] POST /identity/v3/domains => generated 267 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:51:27.748269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235da0e-94a3-4b8b-b344-fae6f2e4ae49 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:27.748872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235da0e-94a3-4b8b-b344-fae6f2e4ae49 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:27.748872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235da0e-94a3-4b8b-b344-fae6f2e4ae49 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:27.750187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 577/1170] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:27.755455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:27.756380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:27.756535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:27.773094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:27.774249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:27.833809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 594/1171] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:27.857035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235da0e-94a3-4b8b-b344-fae6f2e4ae49 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:27.953007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235da0e-94a3-4b8b-b344-fae6f2e4ae49 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:27.984645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:27.985483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:27.985770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:27.986047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] PATH_INFO: `/v3/domains/c4707a7da2de4143ad230069d419f17d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:27.996505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c4707a7da2de4143ad230069d419f17d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:27.997955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ab92f-95f7-44fd-ac8b-0e1d4fdc1da3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:27.999658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 578/1172] 10.222.0.83 () {56 vars in 1183 bytes} [Tue May 10 14:51:27 2022] GET /identity/v3/domains/c4707a7da2de4143ad230069d419f17d => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:28.005263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:28.095888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979ffc-d1ee-470d-a09c-2081389ede3f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:28.126947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:28.127580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:28.127857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:28.128129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:28.128812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:28.130211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f08f7-6d4a-48f3-9095-32c92f5e65b4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:28.153905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 595/1173] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:51:28 2022] POST /identity/v3/projects => generated 361 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:51:30.380606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc018ab-e2ba-467d-ae30-f91251b8f646 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:30.381181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc018ab-e2ba-467d-ae30-f91251b8f646 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:30.381181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc018ab-e2ba-467d-ae30-f91251b8f646 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:30.382550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 579/1174] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:30.387965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:30.388402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:30.388516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:30.404986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:30.406127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:30.466314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 596/1175] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:30.490550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc018ab-e2ba-467d-ae30-f91251b8f646 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:30.584041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc018ab-e2ba-467d-ae30-f91251b8f646 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:30.615689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:30.616785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:30.617067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:30.617341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] PATH_INFO: `/v3/domains/c4707a7da2de4143ad230069d419f17d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:30.618517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c4707a7da2de4143ad230069d419f17d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:30.619930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b688b-1dbf-495e-ac75-88ec368a6d08 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:30.621520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 580/1176] 10.222.0.83 () {56 vars in 1183 bytes} [Tue May 10 14:51:30 2022] GET /identity/v3/domains/c4707a7da2de4143ad230069d419f17d => generated 267 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:30.627217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:30.717697 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f084291-6190-4fc2-9a7f-76484d2c72d9 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:30.749397 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:30.750043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:30.750324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:30.750626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:30.751317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:30.752715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ddf1e-7a4e-4157-aaa0-e886435c2bec None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:30.773041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 597/1177] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:51:30 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:51:32.999601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041c3f-c7d5-49f1-a4bf-6471a7fb8a96 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.000129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041c3f-c7d5-49f1-a4bf-6471a7fb8a96 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.000228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041c3f-c7d5-49f1-a4bf-6471a7fb8a96 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.001517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 581/1178] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:33.007242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.007673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.007780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.024295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:33.025437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:33.086083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 598/1179] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:33.108660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041c3f-c7d5-49f1-a4bf-6471a7fb8a96 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:33.204695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041c3f-c7d5-49f1-a4bf-6471a7fb8a96 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:33.237250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:33.238031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.238316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.238621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.248858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:33.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:33.258382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:51:33.259333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 582/1180] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:33.264304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:33.353788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76177c68-eaf5-4a9e-bc77-3060ca32c30d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:33.385595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:33.386234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.386658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.386938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.387666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:33.387966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:33.389381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:33.396853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 599/1181] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:33.402432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:33.495140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886c4b-bde5-40d0-928e-1a45029eaec4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:33.525888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:33.526761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.527040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.527311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] PATH_INFO: `/v3/users/79205a0b2f674a6c8c823ff2eb11554b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.539941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] RBAC: Authorizing `identity:get_user(user_id=79205a0b2f674a6c8c823ff2eb11554b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:33.541422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:33.543123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 583/1182] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:33 2022] GET /identity/v3/users/79205a0b2f674a6c8c823ff2eb11554b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:33.549184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:33.651032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3a0a-ca1a-4f27-98b1-cfccbef863c0 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:33.681995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:33.682703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.682982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.683255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:33.747620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:33.749091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:33.846071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 600/1183] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:33.854317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:33.957993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a88a2-7cd9-4e7e-a76f-66d37df3e538 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:33.998462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:33.999279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:33.999563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:33.999838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] PATH_INFO: `/v3/projects/d23bbf30142147a5a332078775e9b169` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:34.001007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] RBAC: Authorizing `identity:get_project(project_id=d23bbf30142147a5a332078775e9b169)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:34.002442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74138d-638a-4cd1-90cb-031758d3fcaf None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:34.004121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 584/1184] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:33 2022] GET /identity/v3/projects/d23bbf30142147a5a332078775e9b169 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:34.010122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:34.102258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8f3bd-851b-4678-814c-8c05e330a816 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:34.133817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:34.134487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:34.134761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:34.135032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:34.135832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=d23bbf30142147a5a332078775e9b169, user.id=79205a0b2f674a6c8c823ff2eb11554b) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:34.136134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:34.137540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a5cf-eb19-402a-bb15-2a315a9e85fc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:34.142889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 601/1185] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:51:34 2022] GET /identity/v3/role_assignments?user.id=79205a0b2f674a6c8c823ff2eb11554b&scope.project.id=d23bbf30142147a5a332078775e9b169&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:36.347164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d081-008c-4a22-a1d4-5febebe92c03 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:36.347694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d081-008c-4a22-a1d4-5febebe92c03 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:36.347694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d081-008c-4a22-a1d4-5febebe92c03 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:36.348975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 585/1186] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:36.354683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:36.355107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:36.355238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:36.371289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:36.372449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:36.433180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 602/1187] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:51:36.455035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d081-008c-4a22-a1d4-5febebe92c03 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:36.548419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d081-008c-4a22-a1d4-5febebe92c03 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:36.579685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:36.580447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:36.580723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:36.580991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:36.590995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:36.598491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:36.599615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:51:36.600687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 586/1188] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:36.605596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:36.696094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6b990-0e36-40b9-9967-b52b4fa37bfe None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:36.726934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:36.727576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:36.727859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:36.728133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:36.728856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:36.729159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:36.730523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:36.737949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 603/1189] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:51:36.743751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:36.838523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e023212-e6f3-4a76-a2b6-6f5010c7945e None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:36.869992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139470f-0466-4b96-b623-783f922298ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:36.872090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139470f-0466-4b96-b623-783f922298ad None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:36.872389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139470f-0466-4b96-b623-783f922298ad None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:36.872664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139470f-0466-4b96-b623-783f922298ad None admin] PATH_INFO: `/v3/users/79205a0b2f674a6c8c823ff2eb11554b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:36.874605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139470f-0466-4b96-b623-783f922298ad None admin] RBAC: Authorizing `identity:get_user(user_id=79205a0b2f674a6c8c823ff2eb11554b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:36.876043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139470f-0466-4b96-b623-783f922298ad None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:36.879904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 587/1190] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:36 2022] GET /identity/v3/users/79205a0b2f674a6c8c823ff2eb11554b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:36.886193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:36.979415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d48ab-08b4-4798-8303-7b23733d27fb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:37.011299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:37.012147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:37.012430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:37.012704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] PATH_INFO: `/v3/projects/d23bbf30142147a5a332078775e9b169` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:37.013866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] RBAC: Authorizing `identity:get_project(project_id=d23bbf30142147a5a332078775e9b169)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:37.015310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e2470-d9e9-47ca-b1a9-a0bc8b3b3337 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:37.016980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 604/1191] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:36 2022] GET /identity/v3/projects/d23bbf30142147a5a332078775e9b169 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:37.022826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139470f-0466-4b96-b623-783f922298ad None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:37.113404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139470f-0466-4b96-b623-783f922298ad None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:37.145517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:37.146361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:37.146673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:37.146947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] PATH_INFO: `/v3/projects/d23bbf30142147a5a332078775e9b169/users/79205a0b2f674a6c8c823ff2eb11554b/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:37.148871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] RBAC: Authorizing `identity:create_grant(project_id=d23bbf30142147a5a332078775e9b169, user_id=79205a0b2f674a6c8c823ff2eb11554b, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:37.150288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207210-8a29-4055-a75d-ed4bd04cb330 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:37.162359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 588/1192] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:51:37 2022] PUT /identity/v3/projects/d23bbf30142147a5a332078775e9b169/users/79205a0b2f674a6c8c823ff2eb11554b/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:51:39.359436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc938a5-88f6-45f3-95ae-da2b025e74ee None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:39.359964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc938a5-88f6-45f3-95ae-da2b025e74ee None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:39.359964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc938a5-88f6-45f3-95ae-da2b025e74ee None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:39.361335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 605/1193] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:51:39.369051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:39.369487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:39.369609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:39.386139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:51:39.387349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:39.448987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 589/1194] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:51:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:51:39.472029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc938a5-88f6-45f3-95ae-da2b025e74ee None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:39.572366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc938a5-88f6-45f3-95ae-da2b025e74ee None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:39.602962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:39.603736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:39.604017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:39.604288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:39.614085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:51:39.622484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:51:39.623412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:51:39.624269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 606/1195] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:51:39.629556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:39.720022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b6e92-dc74-4cec-b7b2-ad8ee71052f0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:39.751987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:39.752630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:39.752913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:39.753189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:39.753919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:39.754224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:39.755639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:39.763168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 590/1196] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:39.768674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:39.859893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b82f-ef84-4a44-941c-f2af2938a8bf None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:39.891601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:39.892386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:39.892663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:39.892933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] PATH_INFO: `/v3/users/79205a0b2f674a6c8c823ff2eb11554b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:39.894089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] RBAC: Authorizing `identity:get_user(user_id=79205a0b2f674a6c8c823ff2eb11554b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:39.895518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:39.897642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 607/1197] 10.222.0.83 () {56 vars in 1177 bytes} [Tue May 10 14:51:39 2022] GET /identity/v3/users/79205a0b2f674a6c8c823ff2eb11554b => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:39.903276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:39.995661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38e33-f1e4-4ecf-a894-ecffd52758ab None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:40.026357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:40.027010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:40.027287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:40.027556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:40.088476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:40.089904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:40.179470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 591/1198] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:51:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:51:40.185239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:40.283514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088f48-198c-4cc9-bb7f-ff0e0ca22670 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:40.313761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:40.314621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:40.314897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:40.315161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] PATH_INFO: `/v3/projects/d23bbf30142147a5a332078775e9b169` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:40.316295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] RBAC: Authorizing `identity:get_project(project_id=d23bbf30142147a5a332078775e9b169)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:40.317692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874be17-f3ad-4f78-91c5-927da69547fc None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:40.319912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 608/1199] 10.222.0.83 () {56 vars in 1186 bytes} [Tue May 10 14:51:40 2022] GET /identity/v3/projects/d23bbf30142147a5a332078775e9b169 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:51:40.325903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:40.417414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1a3d-2ee3-4421-9bc6-37e2c28c670c None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:40.448576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:40.449163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:40.449444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:40.449717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:40.450551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=d23bbf30142147a5a332078775e9b169, user.id=79205a0b2f674a6c8c823ff2eb11554b) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:51:40.450858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:40.452274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd24fd5-9f40-4928-9074-2a9f831883c3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:40.457757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 592/1200] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:51:40 2022] GET /identity/v3/role_assignments?user.id=79205a0b2f674a6c8c823ff2eb11554b&scope.project.id=d23bbf30142147a5a332078775e9b169&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:40.692514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301e55-c896-4b3e-9b45-20684129f1f8 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.693133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301e55-c896-4b3e-9b45-20684129f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.693133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301e55-c896-4b3e-9b45-20684129f1f8 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.698322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 609/1201] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:52:40.700492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.700979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.701298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.719767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:40.721011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:40.806441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 593/1202] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:52:40.823478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301e55-c896-4b3e-9b45-20684129f1f8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:40.943643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301e55-c896-4b3e-9b45-20684129f1f8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:40.980773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:40.981869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.982139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.982438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.993329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:40.994795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:41.014281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:41.015566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:41.016621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:41.016621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:41.016621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:41.016621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:41.018611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 610/1203] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:41.022106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:41.144391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfa56-f1c6-4504-b24f-79802161a4c2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:41.183847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:41.184516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:41.184803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:41.185079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:41.185846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:41.186129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:41.187672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb146ce-0d8f-41ee-be25-79216891313f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:41.200812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 594/1204] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:41.207712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:41.323746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978a0d6-16bd-4769-bf79-f50375f7715c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:41.360173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:41.360835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:41.361120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:41.361393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:41.362106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:41.364101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c539d1b-d498-4552-b510-30a6ef16f5a1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:41.388240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 611/1205] 10.222.0.83 () {60 vars in 1132 bytes} [Tue May 10 14:52:41 2022] POST /identity/v3/users => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:43.919496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4389-d6d3-4bdd-9cca-74905d982bdc None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.919970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4389-d6d3-4bdd-9cca-74905d982bdc None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.920066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4389-d6d3-4bdd-9cca-74905d982bdc None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.921646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 595/1206] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:52:43.928680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.929174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.929279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.947130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:43.948315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:44.020765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 612/1207] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:52:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:52:44.043891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4389-d6d3-4bdd-9cca-74905d982bdc None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.147828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4389-d6d3-4bdd-9cca-74905d982bdc None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.185279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d331cb-7740-45c0-b557-098151151283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.185986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d331cb-7740-45c0-b557-098151151283 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.186267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d331cb-7740-45c0-b557-098151151283 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.186570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d331cb-7740-45c0-b557-098151151283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.198004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d331cb-7740-45c0-b557-098151151283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-15d331cb-7740-45c0-b557-098151151283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:52:44.205649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:44.211869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:44.212984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 596/1208] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:44.215066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.321269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663c1e6-f7c4-4ffb-83e6-42e8e8679661 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.355638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.356522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.356811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.357134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.357889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:44.358193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.359794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.369418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 613/1209] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:44.374853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d331cb-7740-45c0-b557-098151151283 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.481491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d331cb-7740-45c0-b557-098151151283 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.520947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.521747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.522166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.522322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.532732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.534286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:44.553053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:44.554038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:44.555132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:44.555132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:44.555132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:44.555132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:44.558589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 597/1210] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:52:44.560082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.663717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1e9f-a2e0-4b52-8e6c-95035742bf8c None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.700096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.700866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.701147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.701487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.702312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:44.702620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.703987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.718288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 614/1211] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:44.724705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.831581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91132932-d0dd-42de-9fdc-89e315c93bd3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.867549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.868373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.868653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.868927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] PATH_INFO: `/v3/users/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.877102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.879404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:52:44.895089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:44.896530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:44.897585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 598/1212] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:44.902108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.008865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd442d8-d86e-4e78-ada4-5a44c411de71 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.049118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.049855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.050143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.050982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.051861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:45.052211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.053733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.075381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 615/1213] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:45.080951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.196131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bd0ae-085d-44f6-89f4-c6c86c1da74f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.232641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.233289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.233652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.233859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.307689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.309612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.421152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 599/1214] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:52:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:52:45.427768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.546067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80454-9ddd-4fc1-b911-b02eee6de098 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.592714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.594077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.594634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.595139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.611275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.612999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:45.632152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:45.633119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:45.634044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:45.634044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:45.634044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:45.634044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:45.634044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 616/1215] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:45.641420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.746956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8eaba-8c35-41a5-8868-a3f9a018b6a3 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.781493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.782151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.782455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.782732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.783435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:45.783739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.785649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.796616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 600/1216] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:45.802589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.907499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289f9ed-857d-4b90-8bbc-9117ad81cf05 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.945280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.946092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.946454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.946784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.955114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.957334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:52:45.972607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:45.973623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:45.974835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 617/1217] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:45.979956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:46.110465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205acc5-2825-4c24-a27a-1b2eb2f038b8 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:46.165784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:46.166909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:46.167394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:46.167870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:46.169308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:46.169846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:46.172457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b50839-d7c1-4f12-92ef-0c816508a6cd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:46.190383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 601/1218] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:52:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:46.196163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:46.294848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47583-903b-4d5b-9c22-1d52ec582cd8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:46.327604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:46.328260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:46.328562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:46.328833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:46.329655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=01c876294f0a400f8bcd833377df454c) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:46.329955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:46.331438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc3918-f078-47a9-88f3-f774394bd66e None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:46.338024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 618/1219] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:52:46 2022] GET /identity/v3/role_assignments?user.id=01c876294f0a400f8bcd833377df454c&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:48.743380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff71e-fb1d-4e27-927f-1c7fa5a7f158 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:48.743380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff71e-fb1d-4e27-927f-1c7fa5a7f158 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:48.745207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff71e-fb1d-4e27-927f-1c7fa5a7f158 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:48.746063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 602/1220] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:52:48.752309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:48.753171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:48.753646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:48.771645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:48.772819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:48.841528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 619/1221] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:52:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:52:48.864647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaff71e-fb1d-4e27-927f-1c7fa5a7f158 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:48.972344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaff71e-fb1d-4e27-927f-1c7fa5a7f158 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.006992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.007712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.008047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.008298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] PATH_INFO: `/v3/roles/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.018409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:52:49.026948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:49.028123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:49.033916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 603/1222] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:52:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:49.034825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.136103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de95c0-6e4f-479a-89ac-fe816a774703 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.170170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.170916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.171292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.171608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.172377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] RBAC: Adding query filter params (name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:49.172695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.174120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.184023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 620/1223] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:49.189403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.301725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f1f42-b8c9-4837-beca-70e72bf3999f None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.338759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.339575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.339855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.340131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.353028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.354521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:49.372706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:49.373681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:49.374672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:49.374672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:49.374672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:49.374672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:49.374672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 604/1224] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:49.381371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.486705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f9cff-84d4-4014-a06a-c3eac23738ee None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.523430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.524085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.524387 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.524635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.525428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:49.525724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.527207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.539396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 621/1225] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:49.545223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.653304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131e8a-98d7-4c2b-9c74-8bf60ca9afa5 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.696739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.697538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.697817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.698085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.705494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.707966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:52:49.724449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:49.725453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:49.726423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 605/1226] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:49.732215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.836045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095258b7-d874-4d1f-8d01-ae5fb291c5a8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.872552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.873339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.873809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.874099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.875086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:49.875433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.877069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.895986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 622/1227] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:49.902315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.009811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233319fa-ade9-4b80-a2c9-e7e5e5e035fb None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.044249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.045155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.045464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.045767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.057526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.059010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:50.077561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:50.079250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:50.080422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:50.080422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:50.080422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:50.080422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:50.080422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 606/1228] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:50.085462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.185170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efede015-bc9e-4e8e-8d3f-91a50e60418b None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.219743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.220399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.220675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.220944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.221647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:50.221944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.223386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.234910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 623/1229] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:50.241816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.340537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1be54-37eb-4386-a598-9d16f5a501db None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.374633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.375431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.375708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.375976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.383162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.384678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:52:50.398417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:50.399507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:50.400560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 607/1230] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:52:50.405479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.507154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe698-d90e-4fbb-b19f-5da083e422c8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.540333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.541417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.541900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.542442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.543834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:50.544330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.546067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1d10-9933-4e63-beb4-24b4bdaf7343 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.558948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 624/1231] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:50.565979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.669037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618df1-66a4-417b-ba70-ca7f1ae43e65 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.704559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.705476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.705764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.706112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/01c876294f0a400f8bcd833377df454c/roles/b69a4cdde6a7450dbdb8be8b08cc39b8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.721240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=01c876294f0a400f8bcd833377df454c, role_id=b69a4cdde6a7450dbdb8be8b08cc39b8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.723334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616f6b5-d400-490c-8a73-72739bd77868 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.737231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 608/1232] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:52:50 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/01c876294f0a400f8bcd833377df454c/roles/b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:53.023492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648fc1-daf2-4c21-a74e-076cc27f7d50 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.024063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648fc1-daf2-4c21-a74e-076cc27f7d50 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.024171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648fc1-daf2-4c21-a74e-076cc27f7d50 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.025555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 625/1233] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:52:53.031685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.032317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.032317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.051277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:53.052850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:53.127435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 609/1234] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:52:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:52:53.153913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e648fc1-daf2-4c21-a74e-076cc27f7d50 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.266312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e648fc1-daf2-4c21-a74e-076cc27f7d50 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.298693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.299483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.299768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.300040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] PATH_INFO: `/v3/roles/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.310683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:52:53.318563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:53.319801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:53.321045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 626/1235] 10.222.0.83 () {56 vars in 1102 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:53.325883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.426767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa6258-7f91-4d53-8b36-6610686512c2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.464425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.465183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.465432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.465734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.466622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:53.466940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.468565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.478507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 610/1236] 10.222.0.83 () {56 vars in 1103 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:53.485261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.587327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7ed0-3ac0-4e9a-9d2c-cce13b342fba None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.621054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.621849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.622127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.622413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.634920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.636479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:53.655343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:53.656525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:53.657643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:53.657643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:53.657643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:53.657643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:53.657643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 627/1237] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:53.664961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.767622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8fd2e-c907-4a59-8ad6-1edbbf03c7e9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.801493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.802249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.802639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.802985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.803736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:53.804043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.805582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.818691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 611/1238] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:53.825176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.936631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cdb2-08b2-4bb1-b518-59d985a67511 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.974283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.975356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.975638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.975977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.984834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.986409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:52:54.003126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:54.004876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:54.006296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 628/1239] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:54.011412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:54.113702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f076dfb-0695-4036-8eec-3e7a8a52b9db None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:54.152902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:54.153682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:54.154255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:54.154616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:54.155433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:54.155779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:54.157272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:54.174579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 612/1240] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:52:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:54.180221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:54.291482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1c578-68c1-4094-9f7f-314f816af0ba None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:54.327400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:54.328301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:54.328619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:54.328980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:54.401946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:54.403896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:54.521544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 629/1241] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:52:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:52:54.523048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:54.647625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03685750-7509-4d4c-8fbe-ca160384fcee None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:54.681201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:54.681991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:54.682271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:54.682574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:54.692702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:54.694145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:54.712014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:54.713086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:54.714002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:54.714002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:54.714002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:54.714002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:54.714002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 613/1242] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:54.719075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:54.821526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2205-76f6-41a3-927e-91600bc3dc90 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:54.855922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:54.856679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:54.857030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:54.857366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:54.858194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:54.858621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:54.860173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:54.872319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 630/1243] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:54.877543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:54.976864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c76b45-63bb-49e3-8f73-b0d737e78029 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:55.014411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:55.015323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:55.015593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:55.015865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:55.022767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:55.024277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:52:55.037399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:55.038514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:55.039446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 614/1244] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:55.044904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:55.188928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437df6-38ae-4ad7-9c6e-b113b5c6ddc1 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:55.224199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:55.224900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:55.225212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:55.225518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:55.226374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:55.226756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:55.228505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3bfdc-b440-498a-8a01-84a9da5a6683 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:55.242640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 631/1245] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:55.248939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:55.351485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34001-07b6-451d-ab72-a767393d39e4 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:55.384861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:55.385517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:55.385799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:55.386076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:55.386929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] RBAC: Adding query filter params (role.id=b69a4cdde6a7450dbdb8be8b08cc39b8, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=01c876294f0a400f8bcd833377df454c) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:55.387237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:55.388675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70837b-e5b4-442a-be14-d404a35af9f0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:55.394674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 615/1246] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:52:55 2022] GET /identity/v3/role_assignments?user.id=01c876294f0a400f8bcd833377df454c&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=b69a4cdde6a7450dbdb8be8b08cc39b8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:57.995924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc2bb-c51d-43fc-94d6-3bd10821a70c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.996456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc2bb-c51d-43fc-94d6-3bd10821a70c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.996456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc2bb-c51d-43fc-94d6-3bd10821a70c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.998246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 632/1247] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:52:58.004865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.005503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.005503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.023044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:58.024230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:58.098809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 616/1248] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:52:58.128718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cc2bb-c51d-43fc-94d6-3bd10821a70c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.238587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cc2bb-c51d-43fc-94d6-3bd10821a70c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.277033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.277829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.278112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.278405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.290078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:52:58.299443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:58.301154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:58.302259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 633/1249] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:58.307557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.414993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa10cc-22ae-423b-8e36-a94521cda076 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.450878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.451553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.451837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.452136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.453026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:58.453328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.454707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.463566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 617/1250] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:58.470288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.583365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca188d8-ec8c-4b80-ae2d-029696651d34 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.618648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.619768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.620053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.620323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.630738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.632200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:58.648744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:58.649847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:58.650956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:58.650956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:58.650956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:58.650956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:58.650956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 634/1251] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:52:58.655757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.755993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceab14-683a-4a0f-9553-5d20c3bd8a58 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.788840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.789500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.789784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.790059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.790810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:58.791115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.792472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.805014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 618/1252] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:58.811591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.919512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8434-83c8-448f-be20-288994fe05b3 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.954176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.955140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.955430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.955706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] PATH_INFO: `/v3/users/placement` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.964538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.965984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:52:58.980665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:58.981912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:52:58.983164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 635/1253] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:58.987874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:59.092363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19147b-0e33-492a-b55e-39294b4a2996 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:59.131148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:59.131941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:59.132251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:59.132619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:59.133660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:59.134120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:59.135767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:59.154033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 619/1254] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:59.159732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:59.264268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8531b3a-4257-4464-9614-e63001084abb None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:59.297852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:59.298576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:59.298864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:59.299160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:59.369030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:59.370509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:59.479917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 636/1255] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:52:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:52:59.481200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:59.585902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca8a8f-4b65-4d69-b18d-2402128f9d19 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:59.619846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:59.620630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:59.620906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:59.621177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:59.630921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:59.632356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:52:59.649692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:59.651170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:59.652690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:52:59.652690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:52:59.652690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:52:59.652690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:59.658425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 620/1256] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:59.658578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:59.771824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65457468-8c61-4daf-bb51-f8d42eac0c31 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:59.806519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:59.807204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:59.807484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:59.807756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:59.808484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:59.808804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:59.810184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:59.822257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 637/1257] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:59.828076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:59.930461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908be4a-0755-49fb-b00c-05ce6778e309 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:59.965240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:59.966044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:59.966324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:59.966624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:59.974048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:59.975539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:52:59.990071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:52:59.991563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:52:59.992515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 621/1258] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:52:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:52:59.997784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:00.098568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616bab8-2fc3-4945-80f8-5a8ad15cc9f8 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:00.132351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:00.133020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:00.133301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:00.133572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:00.134362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:00.134705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:00.136131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60132b3-53b5-4cb0-aad5-521f3b204335 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:00.148820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 638/1259] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:52:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:00.155187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:00.252109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877765-753a-4509-a865-c65bd4a85318 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:00.284101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:00.284750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:00.285031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:00.285303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:00.286119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=01c876294f0a400f8bcd833377df454c) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:00.286460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:00.287880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9b94-1b8a-4b86-8849-83bcf67bcf95 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:00.293378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 622/1260] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:53:00 2022] GET /identity/v3/role_assignments?user.id=01c876294f0a400f8bcd833377df454c&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:02.458068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159fb-5892-40f3-b6be-6fd96d906625 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.458657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159fb-5892-40f3-b6be-6fd96d906625 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.458657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159fb-5892-40f3-b6be-6fd96d906625 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.459990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 639/1261] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:02.465084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.465517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.465648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.482300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:02.483801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:02.566432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 623/1262] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:02.587729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9159fb-5892-40f3-b6be-6fd96d906625 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:02.681447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9159fb-5892-40f3-b6be-6fd96d906625 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:02.712603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:02.713316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.713596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.713867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.723757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:53:02.731404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:02.732520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:02.733588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 640/1263] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:02.740483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:02.831881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765a8ae-5a39-40a9-ac22-c064a066a1f6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:02.863070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:02.863714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.863994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.864266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.864994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:02.865295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:02.866935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:02.874381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 624/1264] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:02.880262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:02.973598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebf625-5b61-44ed-9022-b32cd4256dfc None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.004628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.005396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.005679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.005954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.015734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.017149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:53:03.033701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:03.034724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:03.039417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:53:03.039417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:53:03.039417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:03.039417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:03.039417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 641/1265] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:03.040450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.134578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d69ca-544f-4d04-8c6e-0f28c60494a6 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.165220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.165865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.166144 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.166442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.167174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:03.167476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.168818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.179639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 625/1266] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:53:03.185307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.277762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2347f-4193-4c81-bd11-d52544f19775 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.308281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.309054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.309333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.309604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.317061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.318486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:53:03.332130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:03.333278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:03.334472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 642/1267] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:03.338852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.432029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25637-f88d-424e-8f21-c5ff207cb217 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.463011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.463667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.463947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.464221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.465042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:03.465350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.466822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.482932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 626/1268] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:03.490212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.588871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6c6bb-5b7f-4b35-8c1a-0261234fb5e6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.620594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.621376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.621657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.621930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.631805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.633260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:53:03.658033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:03.659656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:03.660824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:53:03.660824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:53:03.660824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:03.660824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:03.660824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 643/1269] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:03.665626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.760644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701c67-f4f1-4287-b052-677cf3cadc11 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.794045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.794731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.795016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.795292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.796004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:03.796310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.797680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.808766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 627/1270] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:03.814479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.905388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40298d7f-5726-4b7e-a0c5-45bc1c470894 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.936889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.937678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.937955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.938225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.944496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.945901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:53:03.959505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:03.960493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:03.961425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 644/1271] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:03.966519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.060508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417c096-2ece-4169-8680-aaf0ec4ab416 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.091845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.092485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.092769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.093045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.093815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:04.094123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.095573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32df5a6-f6d7-464e-814d-aba1d1d644cf None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.107492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 628/1272] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:04.113721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.205171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5699901-bac6-45dd-a568-7f3928b34bb5 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.237294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.238156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.238460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.238738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] PATH_INFO: `/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/01c876294f0a400f8bcd833377df454c/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.240642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d331a421ec347a8ad32180fbf3f223b, user_id=01c876294f0a400f8bcd833377df454c, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.242311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c39b4-92d7-40d0-865c-7306a91e6234 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.254151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 645/1273] 10.222.0.83 () {58 vars in 1439 bytes} [Tue May 10 14:53:04 2022] PUT /identity/v3/projects/1d331a421ec347a8ad32180fbf3f223b/users/01c876294f0a400f8bcd833377df454c/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:06.396076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41a43-c074-49cd-8654-08f7d1fca0e3 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.396598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41a43-c074-49cd-8654-08f7d1fca0e3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.396598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41a43-c074-49cd-8654-08f7d1fca0e3 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.397975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 629/1274] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:06.403312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.403715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.403863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.419816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:06.420956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:06.482523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 646/1275] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:06.512507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41a43-c074-49cd-8654-08f7d1fca0e3 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:06.609407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41a43-c074-49cd-8654-08f7d1fca0e3 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:06.641911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:06.642715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.642995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.643270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.652938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:53:06.660901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:06.662000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:06.663252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 630/1276] 10.222.0.83 () {56 vars in 1096 bytes} [Tue May 10 14:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:06.668009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:06.759241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a814c57-c70d-4663-80c4-58e8e2496628 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:06.789875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:06.790562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.790840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.791105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.791826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:06.792123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:06.793451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:06.801645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 647/1277] 10.222.0.83 () {56 vars in 1099 bytes} [Tue May 10 14:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:06.807025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:06.898892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f1660-84ea-4c86-83e6-55ff26180603 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:06.931079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:06.931872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.932150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.932421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.942353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:06.943805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:53:06.959804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:06.960880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:06.961990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:53:06.961990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:53:06.961990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:06.961990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:06.961990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 631/1278] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:53:06.966604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.059255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d55ab-8dcc-4650-ab2f-58136459d8e7 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.090061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.090740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.091021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.091293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.092000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:07.092303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.093645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.105047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 648/1279] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:07.110409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.201266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa3f1d-b576-449d-b686-34b5de445afd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.232590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.233369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.233650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.233922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] PATH_INFO: `/v3/users/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.241133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.242588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:53:07.259148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:07.260258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:07.261274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 632/1280] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:07.265959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.356529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856adb41-1e44-4fd7-95dd-d17f334dc549 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.388156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.388802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.389085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.389360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.390184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:07.390524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.391926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.408260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 649/1281] 10.222.0.83 () {56 vars in 1143 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:07.413747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.508755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38fdde-06cb-4fc4-84c0-ab54fb7d4dbd None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.540129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.540762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.541043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.541317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.602655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.604103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.693335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 633/1282] 10.222.0.83 () {58 vars in 1282 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:53:07.699207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.793197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61064b-76b4-4f4c-880d-e1319711edc9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.825693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.826491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.826770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.827040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.836359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.837771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:53:07.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:07.855108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:07.856077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:53:07.856077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:53:07.856077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:53:07.856077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:07.856077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 650/1283] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:07.861279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.959627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c10e1-5050-4c61-a590-8e90ebd41376 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.991998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.992656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.992938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.993211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.993916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:07.994218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.995617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.006815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 634/1284] 10.222.0.83 () {56 vars in 1109 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:08.012400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:08.104052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71671-8a35-45b0-8298-b9966d21f3d9 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:08.135458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:08.136215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:08.136491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:08.136758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:08.142922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:08.144364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:53:08.157562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application with Lock( May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:08.158618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: ERROR keystone.server.flask.application May 10 14:53:08.160359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 651/1285] 10.222.0.83 () {56 vars in 1111 bytes} [Tue May 10 14:53:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:08.164847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:08.257219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e792b01-1c21-48b9-9196-1553e6fa6843 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:08.288541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:08.289178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:08.289457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:08.289735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:08.290522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:08.290832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:08.292238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bbc5d-89a0-4629-adf9-6c7258958c14 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.304968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 635/1286] 10.222.0.83 () {56 vars in 1148 bytes} [Tue May 10 14:53:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:53:08.311141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:08.403964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8be04-5aa2-4c06-9867-7f44ff61bce6 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:08.435227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:08.435866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:08.436144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:08.436412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:08.437220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] RBAC: Adding query filter params (role.id=57e23e25b8c046ba882d07965d852386, scope.project.id=1d331a421ec347a8ad32180fbf3f223b, user.id=01c876294f0a400f8bcd833377df454c) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:08.437526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:08.438953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea1c0d-4dad-4682-a52d-0b21e5770bb4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.444434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 652/1287] 10.222.0.83 () {56 vars in 1374 bytes} [Tue May 10 14:53:08 2022] GET /identity/v3/role_assignments?user.id=01c876294f0a400f8bcd833377df454c&scope.project.id=1d331a421ec347a8ad32180fbf3f223b&role.id=57e23e25b8c046ba882d07965d852386 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:10.670883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea61ba-6c14-423d-b75f-b512b8307e02 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.671427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea61ba-6c14-423d-b75f-b512b8307e02 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.671427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea61ba-6c14-423d-b75f-b512b8307e02 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.672799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 636/1288] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:10.678435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.678836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.678988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.694919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:10.696072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:10.756403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 653/1289] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:10.780002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea61ba-6c14-423d-b75f-b512b8307e02 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.873728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea61ba-6c14-423d-b75f-b512b8307e02 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.904983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.905758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.906075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.906356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.907053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.908418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:10.920552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:10.921790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:10.922893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 637/1290] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:10.927664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:11.016963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb448d4-ae12-488c-892b-36bf4aa3e048 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:11.048190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:11.048823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:11.049101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:11.049369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.050081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:11.050380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:11.051736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53771f4a-d3ec-4d05-8104-d6d08104f881 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:11.056205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 654/1291] 10.222.0.83 () {56 vars in 1116 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:53:11.061681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:11.154646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434e053-72a7-4a1f-92cd-28780e5fc342 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:11.184855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:11.185434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:11.185712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:11.185984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] PATH_INFO: `/v3/services` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.186697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] RBAC: Adding query filter params (type=placement) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:11.186998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:11.188336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d7274-4f14-4bd9-af99-0c78e1707aae None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:11.193060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 638/1292] 10.222.0.83 () {56 vars in 1116 bytes} [Tue May 10 14:53:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:13.421999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f712a-b42f-491b-86f4-57108a4c470c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:13.422550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f712a-b42f-491b-86f4-57108a4c470c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:13.422643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f712a-b42f-491b-86f4-57108a4c470c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:13.423942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 655/1293] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:13.429115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a067a1-73ec-40c3-a1e9-211ddff804b0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:13.429530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a067a1-73ec-40c3-a1e9-211ddff804b0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:13.429725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a067a1-73ec-40c3-a1e9-211ddff804b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:13.446004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f4a067a1-73ec-40c3-a1e9-211ddff804b0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:13.447191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f4a067a1-73ec-40c3-a1e9-211ddff804b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:13.515115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 639/1294] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:13.543073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f712a-b42f-491b-86f4-57108a4c470c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:13.640961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f712a-b42f-491b-86f4-57108a4c470c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:13.672311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:13.672898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:13.673181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:13.673454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:13.674160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:13.675553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e25c4-1e6e-4dc1-8aee-a6cf8af02e12 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:13.687790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 656/1295] 10.222.0.83 () {60 vars in 1141 bytes} [Tue May 10 14:53:13 2022] POST /identity/v3/services => generated 246 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:15.858357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be31e-585b-42a9-be4a-eaec7df6acae None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.858933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be31e-585b-42a9-be4a-eaec7df6acae None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.858933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be31e-585b-42a9-be4a-eaec7df6acae None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.860320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 640/1296] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:15.865554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.866248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.866342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.882758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:15.883912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:15.950362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 657/1297] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:15.972988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be31e-585b-42a9-be4a-eaec7df6acae None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:16.066562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be31e-585b-42a9-be4a-eaec7df6acae None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:16.097109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:16.097875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:16.098150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:16.098436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] PATH_INFO: `/v3/services/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:16.099065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:16.100431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:16.111765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:16.113421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:16.114286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 641/1298] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:53:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:16.119452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:16.209680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671dbd5-b657-41c8-aeda-b152d18fd19d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:16.240389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:16.241272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:16.241550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:16.241820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:16.242529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:16.242828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:16.244152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:16.249240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 658/1299] 10.222.0.83 () {56 vars in 1116 bytes} [Tue May 10 14:53:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:16.255533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:16.347935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d8e9e-1306-48b8-8505-4521608d62db None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:16.378848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:16.379429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:16.379707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:16.379978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:16.380745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] RBAC: Adding query filter params (interface=public, service_id=f4bbd18e90a9499891e4d57d9cf0e4b3, region_id=RegionOne) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:16.381047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:16.382377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5cb9-1564-4ece-b78b-46769b9c12e0 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:16.387082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 642/1300] 10.222.0.83 () {56 vars in 1251 bytes} [Tue May 10 14:53:16 2022] GET /identity/v3/endpoints?service_id=f4bbd18e90a9499891e4d57d9cf0e4b3&interface=public®ion_id=RegionOne => generated 194 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:16.392441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:16.482965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92ade-2ca1-4289-b9da-48c55a29e7c0 None admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:16.516661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:16.517294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:16.517572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:16.517843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:16.518537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:16.519868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e52c3f-3817-4d20-b1d3-e66a53f657c8 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:16.526254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 659/1301] 10.222.0.83 () {56 vars in 1087 bytes} [Tue May 10 14:53:16 2022] GET /identity/v3/services => generated 2191 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:18.692571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a33c-9e99-46d0-b32c-06d62e6b2852 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:18.693135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a33c-9e99-46d0-b32c-06d62e6b2852 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:18.693135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a33c-9e99-46d0-b32c-06d62e6b2852 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:18.694531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 643/1302] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:53:18.699668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:18.700090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:18.700222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:18.716713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:18.717895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:18.778986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 660/1303] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:53:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:53:18.801116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58a33c-9e99-46d0-b32c-06d62e6b2852 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:18.900099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58a33c-9e99-46d0-b32c-06d62e6b2852 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:18.931248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:18.932035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:18.932318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:18.932593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:18.933229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:18.934609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:53:18.946611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:53:18.947771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application with Lock( May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self._enter() May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application return self.creator() May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application created_value = creator( May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: ERROR keystone.server.flask.application May 10 14:53:18.948981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 644/1304] 10.222.0.83 () {56 vars in 1117 bytes} [Tue May 10 14:53:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:53:18.953667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:19.043880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf93f7-3155-481a-a3a9-e9c02af01be0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:19.075545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:19.076190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:19.076471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:19.076742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] PATH_INFO: `/v3/services` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:19.077435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:19.077737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:19.079109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01435f-a71f-4ba1-87e0-f1758bf1fba4 None admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:19.084463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 661/1305] 10.222.0.83 () {56 vars in 1116 bytes} [Tue May 10 14:53:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:19.090033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:19.184492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272c44f-a4ec-48fd-ba3b-6864cdb570c9 None admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:19.215098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:19.215674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:19.215952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:19.216224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] PATH_INFO: `/v3/endpoints` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:19.216936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:19.218277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf04a50-678a-4aa9-9788-03a1bfe4101e None admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:19.235546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 645/1306] 10.222.0.83 () {60 vars in 1144 bytes} [Tue May 10 14:53:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:22.122763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13642-b28a-4cba-960f-af15b9400ded None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.123807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13642-b28a-4cba-960f-af15b9400ded None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.124316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13642-b28a-4cba-960f-af15b9400ded None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.129896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 662/1307] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:22.131239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ba44b-0a6a-4868-baf6-21a55b894198 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.132092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ba44b-0a6a-4868-baf6-21a55b894198 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.132587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ba44b-0a6a-4868-baf6-21a55b894198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.203370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e23ba44b-0a6a-4868-baf6-21a55b894198 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:22.205435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e23ba44b-0a6a-4868-baf6-21a55b894198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.305926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 646/1308] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:22.389352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.390255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.391184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.395058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 663/1309] 10.222.0.83 () {54 vars in 964 bytes} [Tue May 10 14:54:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:22.403572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a56d2-4f49-409b-aef4-1022873f8cc4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.403785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a56d2-4f49-409b-aef4-1022873f8cc4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.404159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a56d2-4f49-409b-aef4-1022873f8cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.496869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0f7a56d2-4f49-409b-aef4-1022873f8cc4 None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:22.498115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0f7a56d2-4f49-409b-aef4-1022873f8cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.576160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 647/1310] 10.222.0.83 () {58 vars in 1061 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:22.586607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.588653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.729545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19246c26-0f66-4a5a-ba78-12c7cba8ad1a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.803591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.804566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.805028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.805481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.807377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.916439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.917482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef128f-aaf9-4914-a5ee-72acf066cc8d service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.936327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 664/1311] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:25.486325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:25.486880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:25.486880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:25.489700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 648/1312] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:25.495646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4652e-cdd7-4309-a432-f0f4e419de68 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:25.496202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4652e-cdd7-4309-a432-f0f4e419de68 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:25.496202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4652e-cdd7-4309-a432-f0f4e419de68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:25.585099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c7c4652e-cdd7-4309-a432-f0f4e419de68 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:25.586323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c7c4652e-cdd7-4309-a432-f0f4e419de68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:25.693735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 665/1313] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:25.731393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:25.732902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:25.836158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a18ff-44ee-49f9-aa89-31e802cca759 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:25.908563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:25.909137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:25.909410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:25.909684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:25.911343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:26.022211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:26.025110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6541b-d642-4750-a863-f196e88a144c service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:26.050835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 649/1314] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:28.589891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:28.590432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:28.590432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:28.591751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 666/1315] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:28.596935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caee6c-94eb-41bf-8a1f-116d27c80ced None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:28.597381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caee6c-94eb-41bf-8a1f-116d27c80ced None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:28.597475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caee6c-94eb-41bf-8a1f-116d27c80ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:28.659562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-48caee6c-94eb-41bf-8a1f-116d27c80ced None None] MFA Rules not processed for user `9b05b527bfb04ddb84873f9672ebcae0`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:28.660772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-48caee6c-94eb-41bf-8a1f-116d27c80ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:28.753293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 650/1316] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:28.793209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:28.795475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:28.896629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e917f4c-d8c2-489c-84c5-bf0015b7062a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:28.966378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:28.967032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:28.967414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:28.967691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:28.971188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:29.073218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:29.074798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41319f91-dc81-4dd4-a777-6687ce4befe9 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:29.091851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 667/1317] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:31.816359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:31.816957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:31.816957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:31.818346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 651/1318] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:31.823656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442a03-28f9-44c5-81bc-963f73d417f2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:31.824095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442a03-28f9-44c5-81bc-963f73d417f2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:31.824203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442a03-28f9-44c5-81bc-963f73d417f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:31.885360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f1442a03-28f9-44c5-81bc-963f73d417f2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:31.886579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f1442a03-28f9-44c5-81bc-963f73d417f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:31.977182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 668/1319] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:32.014628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:32.016643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:32.112482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a5877-f3fd-4485-ac06-f84ec2959983 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:32.180315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:32.180951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:32.181227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:32.181495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:32.183134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:32.284274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:32.285690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc98e96-1a20-4d9e-ad15-a57a139d69b0 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:32.302865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 652/1320] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:35.166972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec615da-63d8-433c-8d94-e16da6417d4a None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.167572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec615da-63d8-433c-8d94-e16da6417d4a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.167572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec615da-63d8-433c-8d94-e16da6417d4a None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.169062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 669/1321] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:54:35.174894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05939578-cde2-4cc8-a5ae-86e74d1d4ad5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.175324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05939578-cde2-4cc8-a5ae-86e74d1d4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.175476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05939578-cde2-4cc8-a5ae-86e74d1d4ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.237548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-05939578-cde2-4cc8-a5ae-86e74d1d4ad5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:35.238769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-05939578-cde2-4cc8-a5ae-86e74d1d4ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:35.331344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 653/1322] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:35.383188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.383473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.383747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.385148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 670/1323] 10.222.0.83 () {54 vars in 964 bytes} [Tue May 10 14:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:35.392371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec0f42-e6cf-4821-b779-e1bb5325cf58 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.392655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec0f42-e6cf-4821-b779-e1bb5325cf58 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.392929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec0f42-e6cf-4821-b779-e1bb5325cf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.464355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-94ec0f42-e6cf-4821-b779-e1bb5325cf58 None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:35.465464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-94ec0f42-e6cf-4821-b779-e1bb5325cf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:35.539864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 654/1324] 10.222.0.83 () {58 vars in 1061 bytes} [Tue May 10 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:35.549716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:35.551358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:35.651122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39e557-6284-4e09-85a5-5babd5005d63 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:35.722371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:35.722966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.723248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.723525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.725137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:35.828637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:35.830070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514bc23-535e-4317-ae3c-5c7eab24f903 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:35.847760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 671/1325] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:38.364284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:38.364862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:38.364862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:38.366464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 655/1326] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:38.371694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f781e-5fb3-446f-a18f-f2df21516d36 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:38.372085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f781e-5fb3-446f-a18f-f2df21516d36 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:38.372251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f781e-5fb3-446f-a18f-f2df21516d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:38.433167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-800f781e-5fb3-446f-a18f-f2df21516d36 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:38.434368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-800f781e-5fb3-446f-a18f-f2df21516d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:38.529511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 672/1327] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:38.565265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:38.566820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:38.664278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a63e0-07d0-40f5-a425-33d7489714e2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:38.733805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:38.735093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:38.735377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:38.735650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:38.737275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:38.849958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:38.851442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e0f2a-c477-40e3-81c5-ee1535a4fe8b service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:38.869689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 656/1328] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:41.152998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:41.153561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:41.153561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:41.154952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 673/1329] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:41.160872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f651-9cba-486f-a4de-46d6fdc3b909 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:41.161320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f651-9cba-486f-a4de-46d6fdc3b909 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:41.161412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f651-9cba-486f-a4de-46d6fdc3b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:41.232078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-aa14f651-9cba-486f-a4de-46d6fdc3b909 None None] MFA Rules not processed for user `9b05b527bfb04ddb84873f9672ebcae0`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:41.233191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-aa14f651-9cba-486f-a4de-46d6fdc3b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:41.325224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 657/1330] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:41.363090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:41.364593 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:41.461978 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d11e41-4be6-4377-ae34-2b0ef3b1d9c4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:41.531139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:41.532224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:41.532508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:41.532785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:41.534405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:41.635847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:41.637292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe80660-25fa-4ca2-baf2-4d2ef68cc25b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:41.655004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 674/1331] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:44.300532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:44.301086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:44.301086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:44.302470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 658/1332] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:44.307985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599a61-7c28-4190-b393-c6062b918ea8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:44.308508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599a61-7c28-4190-b393-c6062b918ea8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:44.308508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599a61-7c28-4190-b393-c6062b918ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:44.378136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f6599a61-7c28-4190-b393-c6062b918ea8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:44.379395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f6599a61-7c28-4190-b393-c6062b918ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:44.479877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 675/1333] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:44.518611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:44.521326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:44.626049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891079a0-b7ff-403c-b529-377a2d5483f5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:44.694109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:44.694722 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:44.695003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:44.695274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:44.696870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:44.798990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:44.800343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1827-9194-4398-a985-d89df7adaec3 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:44.819413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 659/1334] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:47.066088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:47.066675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:47.066675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:47.068054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 676/1335] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:47.073158 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9004-c0dc-45b9-a8d8-b20f438a2af9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:47.073596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9004-c0dc-45b9-a8d8-b20f438a2af9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:47.073690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9004-c0dc-45b9-a8d8-b20f438a2af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:47.136069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-326c9004-c0dc-45b9-a8d8-b20f438a2af9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:47.137279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-326c9004-c0dc-45b9-a8d8-b20f438a2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:47.227776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 660/1336] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:47.267845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:47.269361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:47.367946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ce15b-2291-47cf-bf4f-e01b2696e7fd None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:47.442406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:47.443052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:47.443334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:47.443606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:47.445210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:47.546676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:47.548027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e842e0d-dcce-4184-9021-4cce8f3c5446 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:47.564771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 677/1337] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:50.237342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.237850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.237941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.239316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 661/1338] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:50.244741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69306-b972-4557-b85d-3580f0b3ed70 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.245140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69306-b972-4557-b85d-3580f0b3ed70 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.245296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69306-b972-4557-b85d-3580f0b3ed70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.306560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-11c69306-b972-4557-b85d-3580f0b3ed70 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:50.307739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-11c69306-b972-4557-b85d-3580f0b3ed70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.399039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 678/1339] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:50.434139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:50.435683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.537170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c40a-ec37-483a-b2b2-aa665f90ac98 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:50.605179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:50.605811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.606088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.606356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.607979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.708090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:50.709427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c463e-a1a1-410d-bbea-99ded7c3b28b service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:50.727437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 662/1340] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:54.958986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:54.959496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:54.959496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:54.960784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 679/1341] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:54.965991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a4f-9bd6-4137-975e-bf4ca439c1b2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:54.966442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a4f-9bd6-4137-975e-bf4ca439c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:54.966758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a4f-9bd6-4137-975e-bf4ca439c1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:55.034566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d3d20a4f-9bd6-4137-975e-bf4ca439c1b2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:55.035733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d3d20a4f-9bd6-4137-975e-bf4ca439c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:55.133503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 663/1342] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:55.170309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:55.172984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:55.264629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1ab5d-5d85-491e-b77c-d7e0a39165ef None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:55.268753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:55.269401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:55.269680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:55.269949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:55.278819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:55.280220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3b901-4e6b-47b4-b6e7-6bb711ae2f30 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:55.289217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 680/1343] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:58.106278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c144c2-659c-402b-a227-81facb2cef29 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:58.107174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c144c2-659c-402b-a227-81facb2cef29 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:58.107174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c144c2-659c-402b-a227-81facb2cef29 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:58.108396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 664/1344] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:54:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:54:58.114272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed591e-e01f-4ad7-84b5-ad51aa29faf0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:58.114718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed591e-e01f-4ad7-84b5-ad51aa29faf0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:58.114869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed591e-e01f-4ad7-84b5-ad51aa29faf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:58.141912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b6ed591e-e01f-4ad7-84b5-ad51aa29faf0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:58.143109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b6ed591e-e01f-4ad7-84b5-ad51aa29faf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:58.216034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 681/1345] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:58.253253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c144c2-659c-402b-a227-81facb2cef29 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:58.263648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c144c2-659c-402b-a227-81facb2cef29 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:58.267140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:58.267769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:58.268055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:58.268336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:58.276900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:58.278328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03aa6c-320d-4f49-8965-4281ff98c857 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:58.287436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 665/1346] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:54:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:02.810718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:02.811306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:02.811306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:02.813189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 682/1347] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:02.818325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8282c-7a58-4745-8a97-39cd794cfcd0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:02.818808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8282c-7a58-4745-8a97-39cd794cfcd0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:02.818913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8282c-7a58-4745-8a97-39cd794cfcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:02.835941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-74d8282c-7a58-4745-8a97-39cd794cfcd0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:02.837104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-74d8282c-7a58-4745-8a97-39cd794cfcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:02.845354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 666/1348] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:02.884548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:02.886248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:02.899528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec1382-19ef-4b6c-ad6e-a7eee28e24f7 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:02.902947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:02.903581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:02.903867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:02.904144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:02.913057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:02.914527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930422af-e588-41f6-b7f7-78c5acdc6ad7 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:02.924063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 683/1349] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:08.182137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73c5e-5c27-44ec-bb0a-9bd564a195f7 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:08.183134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73c5e-5c27-44ec-bb0a-9bd564a195f7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:08.183621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73c5e-5c27-44ec-bb0a-9bd564a195f7 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:08.186192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 667/1350] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 14:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:08.192899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085592d-a82f-4192-a2e2-9fc5d43823c9 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:08.193816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085592d-a82f-4192-a2e2-9fc5d43823c9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:08.194439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085592d-a82f-4192-a2e2-9fc5d43823c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:08.343099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9085592d-a82f-4192-a2e2-9fc5d43823c9 None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:08.345720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9085592d-a82f-4192-a2e2-9fc5d43823c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:08.538004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 684/1351] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:12.716820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5c1b-f907-4400-864a-16719d780dd1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:12.717788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5c1b-f907-4400-864a-16719d780dd1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:12.718833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5c1b-f907-4400-864a-16719d780dd1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:12.720336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 668/1352] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 14:55:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:12.725499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4270b6d-40f9-4ecd-8db2-faa35d64a792 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:12.725927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4270b6d-40f9-4ecd-8db2-faa35d64a792 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:12.726242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4270b6d-40f9-4ecd-8db2-faa35d64a792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:12.745812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-a4270b6d-40f9-4ecd-8db2-faa35d64a792 None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:12.747038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a4270b6d-40f9-4ecd-8db2-faa35d64a792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:12.755862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 685/1353] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:14.270045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280682-3242-481c-810f-88de46612a9d None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:14.270635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280682-3242-481c-810f-88de46612a9d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:14.270810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280682-3242-481c-810f-88de46612a9d None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:14.272264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 669/1354] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 14:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:14.277417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b210-d16e-4d71-87e2-4e872fe9324b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:14.277833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b210-d16e-4d71-87e2-4e872fe9324b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:14.277980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b210-d16e-4d71-87e2-4e872fe9324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:14.296116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-04f5b210-d16e-4d71-87e2-4e872fe9324b None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:14.297292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-04f5b210-d16e-4d71-87e2-4e872fe9324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:14.305835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 686/1355] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 14:55:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:18.128606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ea1e-432f-487c-b212-4d26f8a460a2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:18.129177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ea1e-432f-487c-b212-4d26f8a460a2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:18.129470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396ea1e-432f-487c-b212-4d26f8a460a2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:18.131143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 670/1356] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:18.136184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330978-fbab-4114-b758-4ea40a27eb9e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:18.136627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330978-fbab-4114-b758-4ea40a27eb9e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:18.136742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330978-fbab-4114-b758-4ea40a27eb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:18.154592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ad330978-fbab-4114-b758-4ea40a27eb9e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:18.155795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ad330978-fbab-4114-b758-4ea40a27eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:18.164535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 687/1357] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:18.212433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d9610-9064-44e0-aff8-f943b44d1e18 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:18.212718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d9610-9064-44e0-aff8-f943b44d1e18 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:18.212990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d9610-9064-44e0-aff8-f943b44d1e18 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:18.214360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 671/1358] 10.222.0.83 () {54 vars in 961 bytes} [Tue May 10 14:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:18.219713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92b2d-062f-4c43-9a12-2397dc6d799a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:18.220009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92b2d-062f-4c43-9a12-2397dc6d799a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:18.220291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92b2d-062f-4c43-9a12-2397dc6d799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:18.267915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9ec92b2d-062f-4c43-9a12-2397dc6d799a None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:18.269085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9ec92b2d-062f-4c43-9a12-2397dc6d799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:18.348430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 688/1359] 10.222.0.83 () {58 vars in 1058 bytes} [Tue May 10 14:55:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:18.363692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d9610-9064-44e0-aff8-f943b44d1e18 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:18.378670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d9610-9064-44e0-aff8-f943b44d1e18 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:18.383951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:18.384583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:18.384867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:18.385147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:18.393028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:18.394870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1907c8-960d-428b-bc86-a62305d8acc2 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:18.405084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 672/1360] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:20.834011 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba296ed-36ad-468d-ac47-861f9f68603e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:20.834909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba296ed-36ad-468d-ac47-861f9f68603e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:20.835348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba296ed-36ad-468d-ac47-861f9f68603e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:20.836716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 689/1361] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:20.841972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ae8e-1cd9-444c-a595-3181657be549 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:20.842423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ae8e-1cd9-444c-a595-3181657be549 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:20.842551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ae8e-1cd9-444c-a595-3181657be549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:20.860285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-63f6ae8e-1cd9-444c-a595-3181657be549 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:20.861471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-63f6ae8e-1cd9-444c-a595-3181657be549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:20.869435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 673/1362] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:20.912905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba296ed-36ad-468d-ac47-861f9f68603e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:20.930025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba296ed-36ad-468d-ac47-861f9f68603e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:20.935478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:20.936221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:20.936615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:20.937493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:20.945736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:20.947490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462d5c4-1467-43e5-a0d3-03cbe22e5706 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:20.959410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 690/1363] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:23.475930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205a8f2-24bc-4c6d-8143-0b466f914204 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:23.476467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205a8f2-24bc-4c6d-8143-0b466f914204 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:23.476467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205a8f2-24bc-4c6d-8143-0b466f914204 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:23.477836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 674/1364] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:23.483043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aca9d-1ba5-4e9e-b1da-2ad4bcba809b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:23.483478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aca9d-1ba5-4e9e-b1da-2ad4bcba809b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:23.484130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aca9d-1ba5-4e9e-b1da-2ad4bcba809b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:23.502903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6d7aca9d-1ba5-4e9e-b1da-2ad4bcba809b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:23.504080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6d7aca9d-1ba5-4e9e-b1da-2ad4bcba809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:23.512194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 691/1365] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:23.556257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b6d2-fad5-4a58-bb2a-a1e92d9c07ae None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:23.556543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b6d2-fad5-4a58-bb2a-a1e92d9c07ae None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:23.556817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b6d2-fad5-4a58-bb2a-a1e92d9c07ae None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:23.558341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 675/1366] 10.222.0.83 () {54 vars in 961 bytes} [Tue May 10 14:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:23.565681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230e28-dbac-428a-911f-f37968c02aa8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:23.566153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230e28-dbac-428a-911f-f37968c02aa8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:23.566614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230e28-dbac-428a-911f-f37968c02aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:23.584209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0a230e28-dbac-428a-911f-f37968c02aa8 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:23.585677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0a230e28-dbac-428a-911f-f37968c02aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:23.594764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 692/1367] 10.222.0.83 () {58 vars in 1058 bytes} [Tue May 10 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:23.602622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b6d2-fad5-4a58-bb2a-a1e92d9c07ae None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:23.628866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b6d2-fad5-4a58-bb2a-a1e92d9c07ae None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:23.642383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:23.643389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:23.643860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:23.644321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:23.653555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:23.655552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106cfb5-2d83-4f2e-a332-fa39cd1a1307 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:23.665027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 676/1368] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:25.982344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49eec-4436-47d9-9ebb-dcc0b17e0b9f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:25.982887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49eec-4436-47d9-9ebb-dcc0b17e0b9f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:25.982970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49eec-4436-47d9-9ebb-dcc0b17e0b9f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:25.985458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 693/1369] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:25.990965 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0885-7ad5-4361-8175-5fed94dfd7cf None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:25.991649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0885-7ad5-4361-8175-5fed94dfd7cf None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:25.991775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0885-7ad5-4361-8175-5fed94dfd7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.009839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-bd5b0885-7ad5-4361-8175-5fed94dfd7cf None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:26.011302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bd5b0885-7ad5-4361-8175-5fed94dfd7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.019438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 677/1370] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:26.063321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed49eec-4436-47d9-9ebb-dcc0b17e0b9f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:26.079821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed49eec-4436-47d9-9ebb-dcc0b17e0b9f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:26.086267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:26.087054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.087433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.087805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.096704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:26.098366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124156e-dea6-401c-86dd-4de6e744b4b5 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:26.108014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 694/1371] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:26.309177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0703d5-5f74-401f-8d99-febf3cf594b3 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.309730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0703d5-5f74-401f-8d99-febf3cf594b3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.310008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0703d5-5f74-401f-8d99-febf3cf594b3 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.311455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 678/1372] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:26.317540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f428-0b8a-404d-b315-217ba81b982d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.317831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f428-0b8a-404d-b315-217ba81b982d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.318659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f428-0b8a-404d-b315-217ba81b982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.337535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4b63f428-0b8a-404d-b315-217ba81b982d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:26.339088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4b63f428-0b8a-404d-b315-217ba81b982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.348798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 695/1373] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:26.660744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d73a6a-4dfb-4c80-a44b-5e72131bea1d None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.660744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d73a6a-4dfb-4c80-a44b-5e72131bea1d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.660744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d73a6a-4dfb-4c80-a44b-5e72131bea1d None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.661568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 679/1374] 10.222.0.83 () {54 vars in 963 bytes} [Tue May 10 14:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:26.666651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47078c-9e8d-43e6-985c-77086e55ad46 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.667079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47078c-9e8d-43e6-985c-77086e55ad46 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.667452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47078c-9e8d-43e6-985c-77086e55ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.714968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7d47078c-9e8d-43e6-985c-77086e55ad46 None None] MFA Rules not processed for user `f00c2803cc6a4e4a982d05cd419e5301`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:26.716312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7d47078c-9e8d-43e6-985c-77086e55ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.775738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 696/1375] 10.222.0.83 () {58 vars in 1060 bytes} [Tue May 10 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:26.783485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d73a6a-4dfb-4c80-a44b-5e72131bea1d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:26.793985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d73a6a-4dfb-4c80-a44b-5e72131bea1d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:26.798329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:26.799088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.799491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.799866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.808928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:26.810485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec918e-d906-4e95-a88b-3372bee204c8 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:26.820705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 680/1376] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:28.379097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fbdd2-dde4-4d73-98b6-ef15595729bb None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.379665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fbdd2-dde4-4d73-98b6-ef15595729bb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.379665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fbdd2-dde4-4d73-98b6-ef15595729bb None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.381257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 697/1377] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:28.388503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa328b-64d7-4cae-9503-39ecb5c441ea None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.389079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa328b-64d7-4cae-9503-39ecb5c441ea None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.389194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa328b-64d7-4cae-9503-39ecb5c441ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.407795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cafa328b-64d7-4cae-9503-39ecb5c441ea None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:28.409031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cafa328b-64d7-4cae-9503-39ecb5c441ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.417471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 681/1378] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:28.461793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fbdd2-dde4-4d73-98b6-ef15595729bb None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:28.476642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fbdd2-dde4-4d73-98b6-ef15595729bb None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:28.482164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:28.482861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.483297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.484154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.493493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:28.495163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb9c53-a5f9-4b8f-afdb-bff5669b0823 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:28.506722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 698/1379] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:30.761433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd97b-1804-4f3a-9e9c-9d739b439241 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.762011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd97b-1804-4f3a-9e9c-9d739b439241 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.762292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd97b-1804-4f3a-9e9c-9d739b439241 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.763781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 682/1380] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:30.769187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea39d-b621-4f74-8989-d128fae91a45 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.769624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea39d-b621-4f74-8989-d128fae91a45 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.769942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea39d-b621-4f74-8989-d128fae91a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.788652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c38ea39d-b621-4f74-8989-d128fae91a45 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:30.789956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c38ea39d-b621-4f74-8989-d128fae91a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:30.799030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 699/1381] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:30.842063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cd97b-1804-4f3a-9e9c-9d739b439241 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.859278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cd97b-1804-4f3a-9e9c-9d739b439241 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.864685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.865445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.865832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.866206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.874815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.876457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55641db-c924-4ed5-b61c-3bbe56ff8b26 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.886140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 683/1382] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:33.154560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947cd4-7619-4662-8f3c-48b169c63cd3 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:33.155104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947cd4-7619-4662-8f3c-48b169c63cd3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:33.155104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947cd4-7619-4662-8f3c-48b169c63cd3 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:33.156912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 700/1383] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:33.162069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9b10a-5f4f-4077-996d-8b0cf9670f26 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:33.162530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9b10a-5f4f-4077-996d-8b0cf9670f26 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:33.162661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9b10a-5f4f-4077-996d-8b0cf9670f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:33.180478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cda9b10a-5f4f-4077-996d-8b0cf9670f26 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:33.181647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cda9b10a-5f4f-4077-996d-8b0cf9670f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:33.193970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 684/1384] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:33.229954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947cd4-7619-4662-8f3c-48b169c63cd3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:33.245252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947cd4-7619-4662-8f3c-48b169c63cd3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:33.250909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:33.251695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:33.252085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:33.252460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:33.261783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:33.263633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97525e73-8d18-453d-8bb9-b57c7295637c service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:33.275674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 701/1385] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:35.511735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7baf8-9b73-4f39-9f24-4880775fbfaa None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:35.512447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7baf8-9b73-4f39-9f24-4880775fbfaa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:35.512447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7baf8-9b73-4f39-9f24-4880775fbfaa None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:35.517810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 685/1386] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:35.520257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e79d2-c12e-47eb-b8ff-fc211823aea3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:35.520849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e79d2-c12e-47eb-b8ff-fc211823aea3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:35.520849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e79d2-c12e-47eb-b8ff-fc211823aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:35.539379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-565e79d2-c12e-47eb-b8ff-fc211823aea3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:35.540829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-565e79d2-c12e-47eb-b8ff-fc211823aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:35.550625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 702/1387] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:35.601265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7baf8-9b73-4f39-9f24-4880775fbfaa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:35.616387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7baf8-9b73-4f39-9f24-4880775fbfaa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:35.622120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:35.622793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:35.623080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:35.623359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:35.632305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:35.633919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910a3a6-d2d8-432f-9fd2-f9f83fadc080 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:35.643351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 686/1388] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:38.461847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc445-8d62-4bae-a2fb-2ca38c1e97b1 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.462373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc445-8d62-4bae-a2fb-2ca38c1e97b1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.462577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdc445-8d62-4bae-a2fb-2ca38c1e97b1 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.464249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 703/1389] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:38.470947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234bb6-c628-49dd-909d-fa924b808eab None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.471415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234bb6-c628-49dd-909d-fa924b808eab None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.471641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234bb6-c628-49dd-909d-fa924b808eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.492129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ea234bb6-c628-49dd-909d-fa924b808eab None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:38.493329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ea234bb6-c628-49dd-909d-fa924b808eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:38.501551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 687/1390] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:38.546463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdc445-8d62-4bae-a2fb-2ca38c1e97b1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:38.563760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdc445-8d62-4bae-a2fb-2ca38c1e97b1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:38.570353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:38.571153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.571788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.572376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.582969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:38.584663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7663f-88ad-4ced-8c9e-f7fd8b5f3248 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:38.595211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 704/1391] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:41.301483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3cfc0-58f3-469c-b6f8-959ca183c550 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:41.302060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3cfc0-58f3-469c-b6f8-959ca183c550 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:41.302060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3cfc0-58f3-469c-b6f8-959ca183c550 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:41.303340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 688/1392] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:41.308651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b535-e5d1-48d4-8a96-5dd6e3adfe49 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:41.309118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b535-e5d1-48d4-8a96-5dd6e3adfe49 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:41.309238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b535-e5d1-48d4-8a96-5dd6e3adfe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:41.326898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2b28b535-e5d1-48d4-8a96-5dd6e3adfe49 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:41.328647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2b28b535-e5d1-48d4-8a96-5dd6e3adfe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:41.336346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 705/1393] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:41.380903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3cfc0-58f3-469c-b6f8-959ca183c550 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:41.395354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3cfc0-58f3-469c-b6f8-959ca183c550 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:41.401001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:41.401640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:41.401919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:41.402197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:41.410352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:41.411967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ce509-52cd-4d17-8fbb-7578419f1564 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:41.421165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 689/1394] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:42.483835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c0f6-b7f2-4785-8ba0-2eb3f8a418b0 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:42.484386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c0f6-b7f2-4785-8ba0-2eb3f8a418b0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:42.484386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c0f6-b7f2-4785-8ba0-2eb3f8a418b0 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:42.485679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 706/1395] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:42.491079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4b59-f148-4d19-8b66-76fde5ac23b7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:42.491513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4b59-f148-4d19-8b66-76fde5ac23b7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:42.491628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4b59-f148-4d19-8b66-76fde5ac23b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:42.509813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-dd9c4b59-f148-4d19-8b66-76fde5ac23b7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:42.511011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-dd9c4b59-f148-4d19-8b66-76fde5ac23b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:42.518808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 690/1396] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:42.575389 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d095412-0278-49e9-a38f-150796e9b007 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:42.575677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d095412-0278-49e9-a38f-150796e9b007 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:42.575952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d095412-0278-49e9-a38f-150796e9b007 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:42.577314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 707/1397] 10.222.0.83 () {54 vars in 962 bytes} [Tue May 10 14:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:42.582408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2831eeb-a659-4410-a9f9-6c5909e6813c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:42.582705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2831eeb-a659-4410-a9f9-6c5909e6813c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:42.582981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2831eeb-a659-4410-a9f9-6c5909e6813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:42.630969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c2831eeb-a659-4410-a9f9-6c5909e6813c None None] MFA Rules not processed for user `e1bcef95380344929f010f445efb1373`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:42.632157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c2831eeb-a659-4410-a9f9-6c5909e6813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:42.689993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 691/1398] 10.222.0.83 () {58 vars in 1059 bytes} [Tue May 10 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:42.697637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d095412-0278-49e9-a38f-150796e9b007 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:42.707967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d095412-0278-49e9-a38f-150796e9b007 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:42.711532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:42.712267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:42.712613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:42.712953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:42.722381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:42.723823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df65d53-6a57-438a-8275-6ee6ebc0d60f service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:42.732856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 708/1399] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:43.146005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02fca1-2042-4327-a8cf-e1ba8f5e0f52 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.146456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02fca1-2042-4327-a8cf-e1ba8f5e0f52 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.146742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02fca1-2042-4327-a8cf-e1ba8f5e0f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.193250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-df02fca1-2042-4327-a8cf-e1ba8f5e0f52 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:43.194492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-df02fca1-2042-4327-a8cf-e1ba8f5e0f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:43.255716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 692/1400] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:43.299789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040005e4-5011-437a-8f3f-0646ea793a19 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.300158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040005e4-5011-437a-8f3f-0646ea793a19 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.300507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040005e4-5011-437a-8f3f-0646ea793a19 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.301986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 709/1401] 10.222.0.83 () {54 vars in 942 bytes} [Tue May 10 14:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:43.308193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e681-ea04-43e6-b4b7-d65e19ff36d0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.308484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e681-ea04-43e6-b4b7-d65e19ff36d0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.308761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e681-ea04-43e6-b4b7-d65e19ff36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.355782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3409e681-ea04-43e6-b4b7-d65e19ff36d0 None None] MFA Rules not processed for user `2a71e33d66744c208cd0bebb9d09d521`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:43.356963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3409e681-ea04-43e6-b4b7-d65e19ff36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:43.426024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 693/1402] 10.222.0.83 () {58 vars in 1039 bytes} [Tue May 10 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:43.435597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040005e4-5011-437a-8f3f-0646ea793a19 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:43.448710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040005e4-5011-437a-8f3f-0646ea793a19 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:43.453657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:43.454360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.454749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.455093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.463046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:43.464721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ee35b-94c0-40f1-8f10-f86a86fff23a service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:43.475312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 710/1403] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 14:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:55:43.674837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062eef-0ec7-4c90-badb-661ec94b3579 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.675381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062eef-0ec7-4c90-badb-661ec94b3579 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.675381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062eef-0ec7-4c90-badb-661ec94b3579 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.676846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 694/1404] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:43.681795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31b467-bf59-4ed4-b851-eba3e63a4d67 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.682655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31b467-bf59-4ed4-b851-eba3e63a4d67 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.683165 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31b467-bf59-4ed4-b851-eba3e63a4d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.703058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2b31b467-bf59-4ed4-b851-eba3e63a4d67 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:43.704237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2b31b467-bf59-4ed4-b851-eba3e63a4d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:43.711980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 711/1405] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:43.757213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61062eef-0ec7-4c90-badb-661ec94b3579 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:43.773373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61062eef-0ec7-4c90-badb-661ec94b3579 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:43.779981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:43.780774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:43.781147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:43.781515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:43.789916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:43.791635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11007af-0760-4ab5-a8e3-42c423810245 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:43.805324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 695/1406] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:46.109045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae84-1321-4f91-88e4-a10e03180386 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:46.110079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae84-1321-4f91-88e4-a10e03180386 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:46.110634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae84-1321-4f91-88e4-a10e03180386 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:46.112118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 712/1407] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:55:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:55:46.117669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079300-e045-45ad-b965-4fa76276f6b6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:46.118115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079300-e045-45ad-b965-4fa76276f6b6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:46.118404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079300-e045-45ad-b965-4fa76276f6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:46.136923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0d079300-e045-45ad-b965-4fa76276f6b6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:46.141430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0d079300-e045-45ad-b965-4fa76276f6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:46.149415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 696/1408] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:46.193182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae84-1321-4f91-88e4-a10e03180386 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:46.211493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae84-1321-4f91-88e4-a10e03180386 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:46.217102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:46.218091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:46.218492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:46.218827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:46.227406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:46.229038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027969ca-3b0b-41ad-abe7-a6333948223b service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:46.240959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 713/1409] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:06.918889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312a95-2cd6-458f-b91e-4dc354096399 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:06.919354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312a95-2cd6-458f-b91e-4dc354096399 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:06.919457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312a95-2cd6-458f-b91e-4dc354096399 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:06.920873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 697/1410] 10.222.0.83 () {54 vars in 918 bytes} [Tue May 10 14:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:06.926076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e99ce-a78b-4c1b-89af-6b55d1ab0f51 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:06.926561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e99ce-a78b-4c1b-89af-6b55d1ab0f51 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:06.926669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e99ce-a78b-4c1b-89af-6b55d1ab0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:06.944619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-504e99ce-a78b-4c1b-89af-6b55d1ab0f51 None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:06.945793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-504e99ce-a78b-4c1b-89af-6b55d1ab0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:06.954574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 714/1411] 10.222.0.83 () {58 vars in 1015 bytes} [Tue May 10 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:06.968333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532484-6325-44ea-b5da-f6d3f1d09272 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:06.968623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532484-6325-44ea-b5da-f6d3f1d09272 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:06.968900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532484-6325-44ea-b5da-f6d3f1d09272 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:06.970269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 698/1412] 10.222.0.83 () {54 vars in 918 bytes} [Tue May 10 14:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:06.974684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939665-083f-4275-b2e5-34256c2c9876 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:06.974974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939665-083f-4275-b2e5-34256c2c9876 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:06.975251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939665-083f-4275-b2e5-34256c2c9876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:06.993079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-88939665-083f-4275-b2e5-34256c2c9876 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:06.994226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-88939665-083f-4275-b2e5-34256c2c9876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:07.003683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 715/1413] 10.222.0.83 () {58 vars in 1015 bytes} [Tue May 10 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:09.684195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc0e6-6884-4866-b73b-2e3b2ab271cd None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:09.684777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc0e6-6884-4866-b73b-2e3b2ab271cd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:09.684777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc0e6-6884-4866-b73b-2e3b2ab271cd None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:09.686095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:09.691417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476afe-5fc0-4eb6-b568-4bc8f8c42897 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:09.691862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476afe-5fc0-4eb6-b568-4bc8f8c42897 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:09.691970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476afe-5fc0-4eb6-b568-4bc8f8c42897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:09.709462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9d476afe-5fc0-4eb6-b568-4bc8f8c42897 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:09.710681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d476afe-5fc0-4eb6-b568-4bc8f8c42897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:09.719174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:09.760190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc0e6-6884-4866-b73b-2e3b2ab271cd None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:09.774329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc0e6-6884-4866-b73b-2e3b2ab271cd None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:09.779613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:09.780748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:09.781030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:09.781304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:09.789188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:09.790797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c914e3-398d-4029-afc0-71a403a7d125 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:09.800435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 700/1416] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:10.785932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a163a-383e-4db5-a167-5d0b334689a1 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:10.786569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a163a-383e-4db5-a167-5d0b334689a1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:10.786569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a163a-383e-4db5-a167-5d0b334689a1 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:10.787934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:10.793427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d308ee-fb85-43df-afcd-bf45165b8578 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:10.793832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d308ee-fb85-43df-afcd-bf45165b8578 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:10.793972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d308ee-fb85-43df-afcd-bf45165b8578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:10.811316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-16d308ee-fb85-43df-afcd-bf45165b8578 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:10.812490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-16d308ee-fb85-43df-afcd-bf45165b8578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:10.820159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:10.977560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a163a-383e-4db5-a167-5d0b334689a1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:10.991494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a163a-383e-4db5-a167-5d0b334689a1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:10.997045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:10.997672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:10.997957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:10.998235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:11.006108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:11.007720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da383e92-2a8a-4c2d-899a-4f95177d86d7 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:11.017223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 718/1419] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:11.953422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d59e-94b7-4633-848d-50e2a64124f4 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:11.953919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d59e-94b7-4633-848d-50e2a64124f4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:11.954037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d59e-94b7-4633-848d-50e2a64124f4 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:11.955419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:11.960556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15f89-ef92-4b7d-8507-1b8538b68655 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:11.960953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15f89-ef92-4b7d-8507-1b8538b68655 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:11.961150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15f89-ef92-4b7d-8507-1b8538b68655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:11.978838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-18c15f89-ef92-4b7d-8507-1b8538b68655 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:11.980060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-18c15f89-ef92-4b7d-8507-1b8538b68655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:11.987956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:12.151938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d59e-94b7-4633-848d-50e2a64124f4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:12.165610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d59e-94b7-4633-848d-50e2a64124f4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:12.170875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:12.171493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:12.171775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:12.172039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:12.179842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:12.181382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd1e7c-9a8b-4a93-a7d8-eb6e2d6cbbe7 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:12.190847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 703/1422] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:13.104285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a17a4-ae5b-4e66-808d-e31d01d1578e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:13.104861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a17a4-ae5b-4e66-808d-e31d01d1578e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:13.104861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a17a4-ae5b-4e66-808d-e31d01d1578e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:13.106201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:13.111661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f243-c2f2-4273-8e09-d75d7104dbf9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:13.112066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f243-c2f2-4273-8e09-d75d7104dbf9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:13.112202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f243-c2f2-4273-8e09-d75d7104dbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:13.130187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9d87f243-c2f2-4273-8e09-d75d7104dbf9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:13.131380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9d87f243-c2f2-4273-8e09-d75d7104dbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:13.138957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:13.286231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a17a4-ae5b-4e66-808d-e31d01d1578e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:13.300470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a17a4-ae5b-4e66-808d-e31d01d1578e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:13.305625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:13.306242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:13.306548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:13.306822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:13.315089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:13.316671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a180c5a-a99d-4986-821d-292f73949fcf service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:13.325522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 721/1425] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:14.248192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65038009-5bb4-45b1-87b7-1669e35c1518 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:14.249204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65038009-5bb4-45b1-87b7-1669e35c1518 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:14.249312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65038009-5bb4-45b1-87b7-1669e35c1518 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:14.250671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:14.255571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0a00-efb7-4c8e-be09-6792d43485b6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:14.255979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0a00-efb7-4c8e-be09-6792d43485b6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:14.256135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0a00-efb7-4c8e-be09-6792d43485b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:14.273881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-119c0a00-efb7-4c8e-be09-6792d43485b6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:14.275084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-119c0a00-efb7-4c8e-be09-6792d43485b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:14.283208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:14.431062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65038009-5bb4-45b1-87b7-1669e35c1518 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:14.444798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65038009-5bb4-45b1-87b7-1669e35c1518 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:14.450036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:14.450708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:14.450986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:14.451256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:14.459041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:14.460594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c8de4-d48b-49b4-92d0-c8522bbcaabc service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:14.470068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 706/1428] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:15.329597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5946e-09c8-4767-98ef-c9961230249d None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:15.330141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5946e-09c8-4767-98ef-c9961230249d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:15.330141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5946e-09c8-4767-98ef-c9961230249d None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:15.331570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:15.336325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dfa21-f76d-45b1-baa6-1d65698f7e52 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:15.336763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dfa21-f76d-45b1-baa6-1d65698f7e52 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:15.336861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dfa21-f76d-45b1-baa6-1d65698f7e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:15.354339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d34dfa21-f76d-45b1-baa6-1d65698f7e52 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:15.355579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d34dfa21-f76d-45b1-baa6-1d65698f7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:15.363766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:15.511364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5946e-09c8-4767-98ef-c9961230249d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:15.524947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5946e-09c8-4767-98ef-c9961230249d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:15.530135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:15.530787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:15.531066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:15.531337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:15.539467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:15.541036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f948d-3f13-44f2-a8e4-3ba22f357553 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:15.550621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 724/1431] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:16.421719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7156ac-1ecc-4dfe-a20f-a500c693f952 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:16.422252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7156ac-1ecc-4dfe-a20f-a500c693f952 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:16.422252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7156ac-1ecc-4dfe-a20f-a500c693f952 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:16.423687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:16.428412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011016-8a32-4474-8e1f-ccb5a2a78dac None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:16.428859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011016-8a32-4474-8e1f-ccb5a2a78dac None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:16.428954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011016-8a32-4474-8e1f-ccb5a2a78dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:16.446330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2a011016-8a32-4474-8e1f-ccb5a2a78dac None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:16.447525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2a011016-8a32-4474-8e1f-ccb5a2a78dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:16.455043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:16.602658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7156ac-1ecc-4dfe-a20f-a500c693f952 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:16.616991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7156ac-1ecc-4dfe-a20f-a500c693f952 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:16.622552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:16.623175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:16.623457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:16.623750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:16.631826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:16.633390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5f75-8b4e-4aa9-af4e-aa4b8a6db9bc service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:16.642524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 709/1434] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:17.500598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83cd3-044c-4449-8f3b-73a611351d13 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:17.501148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83cd3-044c-4449-8f3b-73a611351d13 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:17.501546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83cd3-044c-4449-8f3b-73a611351d13 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:17.503733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:17.508484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c31c-96d3-48f6-832a-bd432351958c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:17.508919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c31c-96d3-48f6-832a-bd432351958c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:17.509034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c31c-96d3-48f6-832a-bd432351958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:17.527282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f381c31c-96d3-48f6-832a-bd432351958c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:17.528453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f381c31c-96d3-48f6-832a-bd432351958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:17.536196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:17.683585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83cd3-044c-4449-8f3b-73a611351d13 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:17.697810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83cd3-044c-4449-8f3b-73a611351d13 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:17.703617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:17.704269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:17.704555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:17.704828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:17.712721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:17.714289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd2b27-259f-4c8b-b51a-f4c9b49a568b service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:17.723907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 727/1437] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:18.591457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f801-3974-48fa-b540-6b562bdaa3a2 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:18.592000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f801-3974-48fa-b540-6b562bdaa3a2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:18.592000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f801-3974-48fa-b540-6b562bdaa3a2 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:18.593803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:18.598607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44507c-5c1a-4ff9-bda5-4bf9c4733260 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:18.599033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44507c-5c1a-4ff9-bda5-4bf9c4733260 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:18.599154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44507c-5c1a-4ff9-bda5-4bf9c4733260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:18.616993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fd44507c-5c1a-4ff9-bda5-4bf9c4733260 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:18.618445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fd44507c-5c1a-4ff9-bda5-4bf9c4733260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:18.626063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:18.773716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f801-3974-48fa-b540-6b562bdaa3a2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:18.787906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f801-3974-48fa-b540-6b562bdaa3a2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:18.793166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:18.793787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:18.794067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:18.794342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:18.802646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:18.804205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c123a-188a-4098-9fd2-8e0f952abf56 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:18.813093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 712/1440] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:20.895195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d559-50ed-42c2-a387-2a113e1b547c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:20.895720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d559-50ed-42c2-a387-2a113e1b547c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:20.895837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d559-50ed-42c2-a387-2a113e1b547c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:20.897145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:20.902338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0b43-960e-4182-b79e-5eec008c0501 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:20.902801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0b43-960e-4182-b79e-5eec008c0501 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:20.902938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0b43-960e-4182-b79e-5eec008c0501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:20.921350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-52cd0b43-960e-4182-b79e-5eec008c0501 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:20.922570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-52cd0b43-960e-4182-b79e-5eec008c0501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:20.930973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:20.969081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d559-50ed-42c2-a387-2a113e1b547c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:20.979425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d559-50ed-42c2-a387-2a113e1b547c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:20.982991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:20.983621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:20.983902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:20.984175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:20.993002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:20.994405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63481-7129-4bc1-912a-a2d41ec00cd9 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:21.003924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 730/1443] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:23.342751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270fcee-2ac9-45b1-9482-192ee479426c None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:23.343366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270fcee-2ac9-45b1-9482-192ee479426c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:23.343366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270fcee-2ac9-45b1-9482-192ee479426c None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:23.344543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:23.349795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6341-17d5-4bd0-bf15-30f5ce17efcd None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:23.350179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6341-17d5-4bd0-bf15-30f5ce17efcd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:23.350350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6341-17d5-4bd0-bf15-30f5ce17efcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:23.368484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1efc6341-17d5-4bd0-bf15-30f5ce17efcd None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:23.369666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1efc6341-17d5-4bd0-bf15-30f5ce17efcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:23.377427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:56:23.414194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270fcee-2ac9-45b1-9482-192ee479426c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:23.425156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270fcee-2ac9-45b1-9482-192ee479426c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:23.428654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:23.429278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:23.429557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:23.429834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:23.438856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:23.440263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dcbf2-915a-4774-9e3f-2d85d56a31cf service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:23.449255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 715/1446] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:25.943000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26191-0204-4337-98e1-7d5e37a582ee None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:25.943453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26191-0204-4337-98e1-7d5e37a582ee None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:25.943571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26191-0204-4337-98e1-7d5e37a582ee None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:25.944970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:25.950133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd14-bb3a-45e2-b572-10e095d911e9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:25.950600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd14-bb3a-45e2-b572-10e095d911e9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:25.950714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd14-bb3a-45e2-b572-10e095d911e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:25.968764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8315fd14-bb3a-45e2-b572-10e095d911e9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:25.969933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8315fd14-bb3a-45e2-b572-10e095d911e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:25.978303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:26.017096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26191-0204-4337-98e1-7d5e37a582ee None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:26.027575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26191-0204-4337-98e1-7d5e37a582ee None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:26.031072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:26.031701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:26.031982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:26.032256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:26.040842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:26.042241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f1fd-b0a2-497a-9625-7455817f4770 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:26.052062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 733/1449] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:28.370124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5775-a97e-42b4-b75a-bfce8f118947 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:28.370780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5775-a97e-42b4-b75a-bfce8f118947 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:28.370780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5775-a97e-42b4-b75a-bfce8f118947 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:28.372093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:28.377259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dde1e-8315-4c12-a115-bd09bd396a1f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:28.377716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dde1e-8315-4c12-a115-bd09bd396a1f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:28.377823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dde1e-8315-4c12-a115-bd09bd396a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:28.394914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ff8dde1e-8315-4c12-a115-bd09bd396a1f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:28.396078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ff8dde1e-8315-4c12-a115-bd09bd396a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:28.403613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:28.441361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5775-a97e-42b4-b75a-bfce8f118947 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:28.451477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5775-a97e-42b4-b75a-bfce8f118947 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:28.454866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:28.455512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:28.456074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:28.456369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:28.464806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:28.466217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ace71-9f29-49fa-a44b-f2ff2826f6c8 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:28.475729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 718/1452] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:30.799909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114e11c-4973-454f-8d95-01d271129163 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:30.800486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114e11c-4973-454f-8d95-01d271129163 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:30.800486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114e11c-4973-454f-8d95-01d271129163 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:30.801810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:30.807022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0059-311e-4769-b9d7-fe53749fd7f9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:30.807450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0059-311e-4769-b9d7-fe53749fd7f9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:30.807565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0059-311e-4769-b9d7-fe53749fd7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:30.824817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-eddd0059-311e-4769-b9d7-fe53749fd7f9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:30.825972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-eddd0059-311e-4769-b9d7-fe53749fd7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:30.834034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:30.870368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114e11c-4973-454f-8d95-01d271129163 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:30.881277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114e11c-4973-454f-8d95-01d271129163 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:30.884778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:30.885403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:30.885685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:30.885961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:30.895026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:30.896500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f998c2-2a0f-471f-9ad9-20bd995c47d9 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:30.905471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 736/1455] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:33.134187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27940dc6-d4a3-4052-8740-88042109d164 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:33.134776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27940dc6-d4a3-4052-8740-88042109d164 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:33.134776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27940dc6-d4a3-4052-8740-88042109d164 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:33.136186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:33.141978 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a917be-ea35-4036-9106-1c35a950f60a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:33.142413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a917be-ea35-4036-9106-1c35a950f60a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:33.142567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a917be-ea35-4036-9106-1c35a950f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:33.159450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-07a917be-ea35-4036-9106-1c35a950f60a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:33.160622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-07a917be-ea35-4036-9106-1c35a950f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:33.168277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:33.205423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27940dc6-d4a3-4052-8740-88042109d164 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:33.215814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27940dc6-d4a3-4052-8740-88042109d164 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:33.219276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:33.219898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:33.220178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:33.220450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:33.228992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:33.230420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa803f-f8e8-4ffd-a277-702ee1114c56 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:33.240072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 721/1458] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:35.493057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc4e9-6716-4339-91aa-624f8f8fa083 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:35.493638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc4e9-6716-4339-91aa-624f8f8fa083 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:35.493638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc4e9-6716-4339-91aa-624f8f8fa083 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:35.494999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:35.500320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92502-eb7d-4fde-b49d-89013eb1d3ba None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:35.500728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92502-eb7d-4fde-b49d-89013eb1d3ba None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:35.500890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92502-eb7d-4fde-b49d-89013eb1d3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:35.518487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-34f92502-eb7d-4fde-b49d-89013eb1d3ba None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:35.519644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-34f92502-eb7d-4fde-b49d-89013eb1d3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:35.527135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:35.564776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bc4e9-6716-4339-91aa-624f8f8fa083 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:35.575030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bc4e9-6716-4339-91aa-624f8f8fa083 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:35.578562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:35.579199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:35.579484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:35.579759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:35.588445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:35.589855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff04c6-75d5-4207-8f51-09aa6e4657cc service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:35.598924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 739/1461] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:37.828191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716b5-58b9-4af2-8a9f-e122681ce9e1 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:37.828793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716b5-58b9-4af2-8a9f-e122681ce9e1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:37.828793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247716b5-58b9-4af2-8a9f-e122681ce9e1 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:37.830118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:37.835481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d5bc-c7dd-4577-9362-b8eec61eb0e3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:37.835897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d5bc-c7dd-4577-9362-b8eec61eb0e3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:37.836069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d5bc-c7dd-4577-9362-b8eec61eb0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:37.853838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4834d5bc-c7dd-4577-9362-b8eec61eb0e3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:37.855062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4834d5bc-c7dd-4577-9362-b8eec61eb0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:37.863262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:37.900827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247716b5-58b9-4af2-8a9f-e122681ce9e1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:37.911565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247716b5-58b9-4af2-8a9f-e122681ce9e1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:37.915172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:37.915813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:37.916120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:37.916394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:37.926028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:37.927517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6a86-65e6-405a-acf4-958067ca864c service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:37.937192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 724/1464] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:40.241287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c608ecb-69db-41d1-bd6a-461655b0e4fa None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:40.241731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c608ecb-69db-41d1-bd6a-461655b0e4fa None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:40.241931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c608ecb-69db-41d1-bd6a-461655b0e4fa None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:40.244454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:40.249223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02036d-4c40-41fd-9803-c878371b7224 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:40.249695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02036d-4c40-41fd-9803-c878371b7224 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:40.249796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02036d-4c40-41fd-9803-c878371b7224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:40.269902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0f02036d-4c40-41fd-9803-c878371b7224 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:40.271113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0f02036d-4c40-41fd-9803-c878371b7224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:40.278765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:40.315899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c608ecb-69db-41d1-bd6a-461655b0e4fa None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:40.327986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c608ecb-69db-41d1-bd6a-461655b0e4fa None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:40.331519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:40.332162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:40.332445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:40.332720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:40.341477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:40.343752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc013-52f7-4d4c-afb1-5ad0e79236fb service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:40.353182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 742/1467] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:42.590315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e29d23-6201-40b2-b214-303ba38caaa3 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:42.590858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e29d23-6201-40b2-b214-303ba38caaa3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:42.590858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e29d23-6201-40b2-b214-303ba38caaa3 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:42.592696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:56:42.597862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb319a-64ab-42ca-8224-32102c2040f2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:42.598271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb319a-64ab-42ca-8224-32102c2040f2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:42.598441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb319a-64ab-42ca-8224-32102c2040f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:42.616519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-35fb319a-64ab-42ca-8224-32102c2040f2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:56:42.617935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-35fb319a-64ab-42ca-8224-32102c2040f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:42.625586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:56:42.660816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e29d23-6201-40b2-b214-303ba38caaa3 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:42.671765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e29d23-6201-40b2-b214-303ba38caaa3 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:42.675209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:42.675832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:42.676116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:42.676388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:42.684929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:42.686362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27045cd1-a47a-437a-8dab-fa5cfa334ccc service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:42.695547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 727/1470] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:57:15.939989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b796-9125-43c7-b1ca-304c5eefc390 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:15.940532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b796-9125-43c7-b1ca-304c5eefc390 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:15.940532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b796-9125-43c7-b1ca-304c5eefc390 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:15.941903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 744/1471] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:15.947145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266919d-5634-43f7-a8a1-628942455db4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:15.947599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266919d-5634-43f7-a8a1-628942455db4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:15.947707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266919d-5634-43f7-a8a1-628942455db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:15.964814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f266919d-5634-43f7-a8a1-628942455db4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:15.966004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f266919d-5634-43f7-a8a1-628942455db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:15.974088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 728/1472] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:57:16.020371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4ea5-2b5b-4f5d-9c8b-7eea9908b876 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:16.020662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4ea5-2b5b-4f5d-9c8b-7eea9908b876 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:16.020931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4ea5-2b5b-4f5d-9c8b-7eea9908b876 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:16.022266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 745/1473] 10.222.0.83 () {54 vars in 962 bytes} [Tue May 10 14:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:16.027098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd356-53dd-4607-bba6-60f73e347ddc None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:16.027383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd356-53dd-4607-bba6-60f73e347ddc None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:16.027655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dd356-53dd-4607-bba6-60f73e347ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:16.045819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d72dd356-53dd-4607-bba6-60f73e347ddc None None] MFA Rules not processed for user `e1bcef95380344929f010f445efb1373`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:16.047034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d72dd356-53dd-4607-bba6-60f73e347ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:16.053329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 729/1474] 10.222.0.83 () {58 vars in 1059 bytes} [Tue May 10 14:57:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:57:16.060373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b4ea5-2b5b-4f5d-9c8b-7eea9908b876 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:16.070444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b4ea5-2b5b-4f5d-9c8b-7eea9908b876 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:16.073798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:16.074430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:16.074712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:16.074983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:16.083842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:16.085227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53c140-8821-4521-bab9-79c90902defc service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:16.094757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 746/1475] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:57:18.380739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce104-e66b-44f3-828c-b4921628ee6a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:18.381295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce104-e66b-44f3-828c-b4921628ee6a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:18.381295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce104-e66b-44f3-828c-b4921628ee6a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:18.382674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 730/1476] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:18.387760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf478-e2bc-49ab-b35e-65b9b88765c4 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:18.388207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf478-e2bc-49ab-b35e-65b9b88765c4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:18.388331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf478-e2bc-49ab-b35e-65b9b88765c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:18.405071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-a73bf478-e2bc-49ab-b35e-65b9b88765c4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:18.406235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a73bf478-e2bc-49ab-b35e-65b9b88765c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:18.413735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 747/1477] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:57:18.436927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ce104-e66b-44f3-828c-b4921628ee6a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:18.449226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ce104-e66b-44f3-828c-b4921628ee6a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:18.453683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:18.454322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:18.454639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:18.454912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:18.455626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:18.457345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:57:18.457345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 14:57:18.457345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 14:57:18.458506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b33bd-3b3f-4e4f-8d78-89a73b2babe1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:18.476663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 731/1478] 10.222.0.83 () {56 vars in 1108 bytes} [Tue May 10 14:57:18 2022] GET /identity/v3/projects => generated 2553 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:57:20.687142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1906-a974-4198-b153-15065f46b93b None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:20.687737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1906-a974-4198-b153-15065f46b93b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:20.687737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1906-a974-4198-b153-15065f46b93b None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:20.689055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 748/1479] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:20.694166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78640d4-400a-4771-ae55-0dba824c38a5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:20.694648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78640d4-400a-4771-ae55-0dba824c38a5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:20.694755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78640d4-400a-4771-ae55-0dba824c38a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:20.713142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b78640d4-400a-4771-ae55-0dba824c38a5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:20.714331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b78640d4-400a-4771-ae55-0dba824c38a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:20.722610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 732/1480] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:57:20.764236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1906-a974-4198-b153-15065f46b93b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:20.777797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1906-a974-4198-b153-15065f46b93b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:20.783636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:20.784441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:20.784545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:20.784786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:20.792510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:20.794057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52787e75-4694-44f7-8933-513f7f3e4441 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:20.802962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 749/1481] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:57:23.153456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afe69-93d6-40d4-9dfb-a09075946472 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:23.154028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afe69-93d6-40d4-9dfb-a09075946472 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:23.154028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afe69-93d6-40d4-9dfb-a09075946472 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:23.155439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 733/1482] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:23.160584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115c87f-2fd8-46d3-b65c-153af11c3e69 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:23.160993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115c87f-2fd8-46d3-b65c-153af11c3e69 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:23.161129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115c87f-2fd8-46d3-b65c-153af11c3e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:23.178046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3115c87f-2fd8-46d3-b65c-153af11c3e69 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:23.179171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3115c87f-2fd8-46d3-b65c-153af11c3e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:23.187313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 750/1483] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:57:23.222224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0afe69-93d6-40d4-9dfb-a09075946472 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:23.232590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0afe69-93d6-40d4-9dfb-a09075946472 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:23.236110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:23.236727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:23.237033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:23.237304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:23.245989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:23.247952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8205f2-a1cc-4106-b20f-b417d72f4e15 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:23.257180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 734/1484] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:57:25.510081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cdd74-edea-4d4a-904e-647b4ae92b0f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:25.510703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cdd74-edea-4d4a-904e-647b4ae92b0f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:25.510703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cdd74-edea-4d4a-904e-647b4ae92b0f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:25.512054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 751/1485] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:25.517738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425621-fb72-4ea9-ac19-d8feb63d4979 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:25.518189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425621-fb72-4ea9-ac19-d8feb63d4979 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:25.518308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425621-fb72-4ea9-ac19-d8feb63d4979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:25.537382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5e425621-fb72-4ea9-ac19-d8feb63d4979 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:25.538600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5e425621-fb72-4ea9-ac19-d8feb63d4979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:25.546922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 735/1486] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:57:25.584158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cdd74-edea-4d4a-904e-647b4ae92b0f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:25.594843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cdd74-edea-4d4a-904e-647b4ae92b0f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:25.598240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:25.598896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:25.599178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:25.599452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:25.608798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:25.610195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54cda2-25ff-47d5-85ec-2d113ff1e875 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:25.619401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 752/1487] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:57:27.967149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6baea3-00d6-4aa6-bcca-0f8b4a8fce9a None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:27.967690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6baea3-00d6-4aa6-bcca-0f8b4a8fce9a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:27.967690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6baea3-00d6-4aa6-bcca-0f8b4a8fce9a None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:27.969066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 736/1488] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:27.974315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792fc0-ec21-4d80-ba31-4ced7f45071b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:27.974768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792fc0-ec21-4d80-ba31-4ced7f45071b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:27.974888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792fc0-ec21-4d80-ba31-4ced7f45071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:27.993237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-79792fc0-ec21-4d80-ba31-4ced7f45071b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:27.996570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-79792fc0-ec21-4d80-ba31-4ced7f45071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:28.004731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 753/1489] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:57:28.044119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6baea3-00d6-4aa6-bcca-0f8b4a8fce9a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:28.058363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6baea3-00d6-4aa6-bcca-0f8b4a8fce9a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:28.062708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:28.063388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:28.063666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:28.063939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:28.073363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:28.074821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16151382-75b4-4a0d-90ea-8d04eed04484 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:28.083843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 737/1490] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:57:30.491348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe510dc-3998-4eac-8716-52c1591df65c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:30.491909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe510dc-3998-4eac-8716-52c1591df65c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:30.491909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe510dc-3998-4eac-8716-52c1591df65c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:30.493285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 754/1491] 10.222.0.83 () {54 vars in 926 bytes} [Tue May 10 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:57:30.498732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58823b49-e8a5-46a3-aee8-226b7fda3d28 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:30.499140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58823b49-e8a5-46a3-aee8-226b7fda3d28 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:30.499295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58823b49-e8a5-46a3-aee8-226b7fda3d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:30.517299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-58823b49-e8a5-46a3-aee8-226b7fda3d28 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:57:30.518468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-58823b49-e8a5-46a3-aee8-226b7fda3d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:57:30.526087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 738/1492] 10.222.0.83 () {58 vars in 1023 bytes} [Tue May 10 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:57:30.564267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe510dc-3998-4eac-8716-52c1591df65c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:57:30.574641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe510dc-3998-4eac-8716-52c1591df65c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:57:30.578658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:57:30.579291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:57:30.579573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:57:30.579867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:57:30.588412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:57:30.589830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cb62b-44de-40fc-a3c5-bd2abc0588ee service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:57:30.599379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 755/1493] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:58:57.338312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.338902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.338902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.358102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:58:57.359309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:57.366973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 739/1494] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:58:57.376605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.377098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.377208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.394047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:58:57.395228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:57.402886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 756/1495] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:58:57.407126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.416279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef884e-78d7-420a-b899-ac2896208d99 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.421191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.421826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.422106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.422378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.423109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:58:57.423409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.424410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.424410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 14:58:57.424410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 14:58:57.425285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.437050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 740/1496] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:58:57.444802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.453408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9fa8c-41ad-4a2c-bcbe-4fb0abb4e8d0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.457822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.458552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.458835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.459106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.459780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.461509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.461509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 14:58:57.461509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 14:58:57.462723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.500666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 757/1497] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:58:57.501859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.511351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab5a3-db32-4b5a-a387-51163ce143c3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.568762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.569422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.569703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.569977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.570692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.571696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.571696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 14:58:57.571696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 14:58:57.572882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.592351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 741/1498] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:58:57.596349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.606638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c4f85-697e-4b90-8bb9-628621b8bb73 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.610951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.611614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.611896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.612149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.612855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.613844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.613844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 14:58:57.613844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 14:58:57.614816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.627843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 758/1499] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:58:57.631568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.641079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8480fd8-f0fc-452d-8317-b5f220b3e3c3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.645631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.646283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.646586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.646862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.647569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.648626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.648626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 14:58:57.648626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 14:58:57.649490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2eb8-f6ea-49e9-aeb6-953b69478b87 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.662887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 742/1500] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:58:57.666850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.677212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627ad2-0749-4d7f-a20c-c89950d91cbd admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.681696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.682597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.682874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.683154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] PATH_INFO: `/v3/projects/690ba1a449c24f13908b11e1854008d3/users/ea68b1aab29740748137c6f56e3e5f6e/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.697741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690ba1a449c24f13908b11e1854008d3, user_id=ea68b1aab29740748137c6f56e3e5f6e, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.699006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:57.699006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 14:58:57.699006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 14:58:57.700906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2a73e-d6fc-423d-99a3-82ee46cc0fc0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.714254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 759/1501] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 14:58:57 2022] PUT /identity/v3/projects/690ba1a449c24f13908b11e1854008d3/users/ea68b1aab29740748137c6f56e3e5f6e/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:58:57.725934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb167bd-b572-4a47-a61b-281459f36c7b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.726333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb167bd-b572-4a47-a61b-281459f36c7b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.726820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb167bd-b572-4a47-a61b-281459f36c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.752710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8eb167bd-b572-4a47-a61b-281459f36c7b None None] MFA Rules not processed for user `ea68b1aab29740748137c6f56e3e5f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:58:57.753891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8eb167bd-b572-4a47-a61b-281459f36c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:57.814226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 743/1502] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:58:57.827380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf6f-776e-459c-b7b4-0244144d3b66 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.827672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf6f-776e-459c-b7b4-0244144d3b66 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.827954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf6f-776e-459c-b7b4-0244144d3b66 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.829315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 760/1503] 10.222.0.83 () {54 vars in 963 bytes} [Tue May 10 14:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:58:57.834103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4307-7a5e-4deb-b842-08929455338d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.834417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4307-7a5e-4deb-b842-08929455338d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.834728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4307-7a5e-4deb-b842-08929455338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.853072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-eb6e4307-7a5e-4deb-b842-08929455338d None None] MFA Rules not processed for user `f00c2803cc6a4e4a982d05cd419e5301`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:58:57.854249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-eb6e4307-7a5e-4deb-b842-08929455338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:57.901801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 744/1504] 10.222.0.83 () {58 vars in 1060 bytes} [Tue May 10 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:58:57.910085 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29faf6f-776e-459c-b7b4-0244144d3b66 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:57.921900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29faf6f-776e-459c-b7b4-0244144d3b66 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:57.925388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:57.926016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.926298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.926608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.934600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:57.936003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:57.944364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 761/1505] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:58:57.985585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5e7-2737-4a32-b892-1a1eab44bc65 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:57.985879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5e7-2737-4a32-b892-1a1eab44bc65 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:57.986169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf5e7-2737-4a32-b892-1a1eab44bc65 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:57.987595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 745/1506] 10.222.0.83 () {52 vars in 829 bytes} [Tue May 10 14:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:58:57.991080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:58.049459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56449750-8dc6-419a-8881-fecc96dd9b71 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:58.053861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:58.054744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:58.055025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:58.055298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] PATH_INFO: `/v3/users/ea68b1aab29740748137c6f56e3e5f6e` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:58.056485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea68b1aab29740748137c6f56e3e5f6e)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:58.057511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:58.057511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 14:58:58.057511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 14:58:58.058779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:58.120071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea68b1aab29740748137c6f56e3e5f6e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.120815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea68b1aab29740748137c6f56e3e5f6e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.132602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea68b1aab29740748137c6f56e3e5f6e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.133343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea68b1aab29740748137c6f56e3e5f6e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.142528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea68b1aab29740748137c6f56e3e5f6e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.165870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 762/1507] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 14:58:57 2022] DELETE /identity/v3/users/ea68b1aab29740748137c6f56e3e5f6e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:58:58.170279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:58.170592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:58.170872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:58.187408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:58:58.188595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:58.244150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 746/1508] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:58:58.254318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:58.255707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:58.311762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6c18c-1b53-42ef-a349-91aed369c44e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:58.315300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:58.316219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:58.316219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:58.316473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:58.325936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:58.327392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0b949-361b-476e-b64f-54601149b8f9 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:58.336457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 763/1509] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:58:58.656624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:58:58.658095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:58:58.669465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af570-17bf-4e4f-88c3-d781e4d58eec None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:58:58.674071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:58:58.674972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:58:58.675268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:58:58.675523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] PATH_INFO: `/v3/projects/690ba1a449c24f13908b11e1854008d3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:58:58.676762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=690ba1a449c24f13908b11e1854008d3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:58:58.677708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:58:58.677708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 14:58:58.677708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 14:58:58.679001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:58:58.749240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690ba1a449c24f13908b11e1854008d3', 'request_id': 'req-89f9a230-5b0e-49b6-9612-3f6ad4686baf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.755049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690ba1a449c24f13908b11e1854008d3', 'request_id': 'req-89f9a230-5b0e-49b6-9612-3f6ad4686baf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.768559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690ba1a449c24f13908b11e1854008d3', 'request_id': 'req-89f9a230-5b0e-49b6-9612-3f6ad4686baf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.784821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] The token cache is being invalidate because project 690ba1a449c24f13908b11e1854008d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:58:58.785084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.785791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-89f9a230-5b0e-49b6-9612-3f6ad4686baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:58:58.787641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 747/1510] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 14:58:58 2022] DELETE /identity/v3/projects/690ba1a449c24f13908b11e1854008d3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:00:47.739830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:47.740427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:47.740427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:47.741794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 15:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:00:47.747601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432569f5-adea-4f1e-8b51-52eae811999d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:47.748013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432569f5-adea-4f1e-8b51-52eae811999d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:47.748156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432569f5-adea-4f1e-8b51-52eae811999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:47.767623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-432569f5-adea-4f1e-8b51-52eae811999d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:00:47.768812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-432569f5-adea-4f1e-8b51-52eae811999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:00:47.824288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:00:47.993003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:00:47.994619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:00:48.064806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d5fc-f110-4d8e-9140-f6b982d68498 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:00:48.070065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:00:48.070719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:48.071004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:48.071280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:48.079091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:00:48.080662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38533b22-9cac-4b0e-b300-ee195789aca7 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:00:48.089671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 765/1513] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:00:48.992901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60cfc-6049-43d2-837d-3d478039a0da None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:48.993464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60cfc-6049-43d2-837d-3d478039a0da None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:48.993464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60cfc-6049-43d2-837d-3d478039a0da None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:48.994853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 15:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:00:49.000755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7810-69f0-4030-b769-991b4ffd5f2a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:49.001162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7810-69f0-4030-b769-991b4ffd5f2a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:49.001349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7810-69f0-4030-b769-991b4ffd5f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:49.019575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-605b7810-69f0-4030-b769-991b4ffd5f2a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:00:49.020790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-605b7810-69f0-4030-b769-991b4ffd5f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:00:49.029077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:00:49.179494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf60cfc-6049-43d2-837d-3d478039a0da None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:00:49.193786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf60cfc-6049-43d2-837d-3d478039a0da None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:00:49.199237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:00:49.200490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:49.200773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:49.201044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:49.209556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:00:49.211770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745bf7-609e-4f15-87ca-b1e247cbd9ef service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:00:49.221593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 750/1516] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:00:51.248167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcd437-0faf-43d1-b3f6-de4d2a4b08b8 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:51.248704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcd437-0faf-43d1-b3f6-de4d2a4b08b8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:51.248704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcd437-0faf-43d1-b3f6-de4d2a4b08b8 None None] PATH_INFO: `/v3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:51.249999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 767/1517] 10.222.0.87 () {54 vars in 925 bytes} [Tue May 10 15:00:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:00:51.260800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b076-24ba-4d92-9168-17782d8eb288 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:51.261231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b076-24ba-4d92-9168-17782d8eb288 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:51.261334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b076-24ba-4d92-9168-17782d8eb288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:51.279929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1c75b076-24ba-4d92-9168-17782d8eb288 None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:00:51.281108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1c75b076-24ba-4d92-9168-17782d8eb288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:00:51.350019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 751/1518] 10.222.0.87 () {58 vars in 1015 bytes} [Tue May 10 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:00:55.826761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dab74-42e2-4220-a2a3-6930cbbf6ab3 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:55.827291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dab74-42e2-4220-a2a3-6930cbbf6ab3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:55.827291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dab74-42e2-4220-a2a3-6930cbbf6ab3 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:55.828612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 768/1519] 10.222.0.83 () {54 vars in 960 bytes} [Tue May 10 15:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:00:55.833384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade156-3c76-4770-a79a-f3902a549dff None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:55.833795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade156-3c76-4770-a79a-f3902a549dff None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:55.833934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade156-3c76-4770-a79a-f3902a549dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:55.851676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d5ade156-3c76-4770-a79a-f3902a549dff None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:00:55.852844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d5ade156-3c76-4770-a79a-f3902a549dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:00:55.861810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 752/1520] 10.222.0.83 () {58 vars in 1057 bytes} [Tue May 10 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:00:55.868933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dab74-42e2-4220-a2a3-6930cbbf6ab3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:00:55.882049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dab74-42e2-4220-a2a3-6930cbbf6ab3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:00:55.886865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:00:55.887486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:00:55.887767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:00:55.888039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:00:55.895969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:00:55.897523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230006-80cd-48d8-a98b-03afe1d92949 service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:00:55.907260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 769/1521] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:00:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:00.518571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd19b-a6dc-4f2d-923d-0a03b6146bce None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:00.519131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd19b-a6dc-4f2d-923d-0a03b6146bce None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:00.519131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd19b-a6dc-4f2d-923d-0a03b6146bce None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:00.520477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 15:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:00.526440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad493b-b23f-4d89-b646-c1d88c64ea94 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:00.527989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad493b-b23f-4d89-b646-c1d88c64ea94 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:00.528401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad493b-b23f-4d89-b646-c1d88c64ea94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:00.553449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d2ad493b-b23f-4d89-b646-c1d88c64ea94 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:00.554683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d2ad493b-b23f-4d89-b646-c1d88c64ea94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:00.562361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 15:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:00.716682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd19b-a6dc-4f2d-923d-0a03b6146bce None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:00.731110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd19b-a6dc-4f2d-923d-0a03b6146bce None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:00.736460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:00.737087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:00.737365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:00.737638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:00.745671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:00.747857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c03084-2d0b-4022-8b97-45ad7de8bc9a service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:00.756905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 754/1524] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:01.636996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50a30-c7a2-4b57-9dae-9fcb98294536 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:01.637548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50a30-c7a2-4b57-9dae-9fcb98294536 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:01.637548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50a30-c7a2-4b57-9dae-9fcb98294536 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:01.638968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 15:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:01.643724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c58f-9438-4059-a1d1-5262f60ca1f4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:01.644189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c58f-9438-4059-a1d1-5262f60ca1f4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:01.644294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c58f-9438-4059-a1d1-5262f60ca1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:01.662178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-31d4c58f-9438-4059-a1d1-5262f60ca1f4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:01.663370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-31d4c58f-9438-4059-a1d1-5262f60ca1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:01.671887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:01.823857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf50a30-c7a2-4b57-9dae-9fcb98294536 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:01.837698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf50a30-c7a2-4b57-9dae-9fcb98294536 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:01.843495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:01.844120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:01.844401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:01.844674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:01.852535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:01.854098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60121-0965-489d-b012-1adfe16da294 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:01.863777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 772/1527] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:05.961490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:05.962016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:05.962138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:05.963444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 15:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:05.968481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d07e8-fad8-4b05-92f1-3a7c13808fd2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:05.968892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d07e8-fad8-4b05-92f1-3a7c13808fd2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:05.969040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d07e8-fad8-4b05-92f1-3a7c13808fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:05.987047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d51d07e8-fad8-4b05-92f1-3a7c13808fd2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:05.988226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d51d07e8-fad8-4b05-92f1-3a7c13808fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:05.996718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:06.031649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:06.033296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:06.088687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129220d-281c-4ea2-9ad5-34f832a0b271 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:06.092236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:06.092879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:06.093164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:06.093442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:06.102100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:06.103572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28012338-a78d-447b-a275-7a8e9674dcf1 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:06.113303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 757/1530] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:08.315538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013bb8-49ef-4539-8542-29df6fa06b7e None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:08.316113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013bb8-49ef-4539-8542-29df6fa06b7e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:08.316113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013bb8-49ef-4539-8542-29df6fa06b7e None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:08.317457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 15:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:08.322523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561543a3-9d9a-484b-8526-4058d6f84c2b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:08.322999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561543a3-9d9a-484b-8526-4058d6f84c2b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:08.323111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561543a3-9d9a-484b-8526-4058d6f84c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:08.341418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-561543a3-9d9a-484b-8526-4058d6f84c2b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:08.342628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-561543a3-9d9a-484b-8526-4058d6f84c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:08.351169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:08.385678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3013bb8-49ef-4539-8542-29df6fa06b7e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:08.396620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3013bb8-49ef-4539-8542-29df6fa06b7e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:08.400164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:08.400794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:08.401075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:08.401349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:08.410543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:08.411957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d752ac-56f9-4797-82b0-bf994b0a95cf service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:08.421018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 775/1533] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:12.241822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:12.242347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:12.242347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:12.243709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:12.249625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd53f-969f-441d-b6c5-8f26517d1c78 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:12.250034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd53f-969f-441d-b6c5-8f26517d1c78 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:12.250177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd53f-969f-441d-b6c5-8f26517d1c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:12.268248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-834fd53f-969f-441d-b6c5-8f26517d1c78 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:12.269426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-834fd53f-969f-441d-b6c5-8f26517d1c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:12.277154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:12.547388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:12.549030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:12.603470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b312c10-9df5-4336-a7eb-db9c6104e653 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:12.606958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:12.607599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:12.607879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:12.608150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:12.616733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:12.618464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b4629-b9f1-4238-a0ab-1ad7a8528502 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:12.627746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 760/1536] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:14.891462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662091fc-d107-4af9-84b6-d15397652dab None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:14.892043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662091fc-d107-4af9-84b6-d15397652dab None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:14.892043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662091fc-d107-4af9-84b6-d15397652dab None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:14.893373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 15:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:01:14.898656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bbeb-0bab-48f3-91f4-4e2d122cf0d1 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:14.899083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bbeb-0bab-48f3-91f4-4e2d122cf0d1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:14.899286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bbeb-0bab-48f3-91f4-4e2d122cf0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:14.917436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d732bbeb-0bab-48f3-91f4-4e2d122cf0d1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:01:14.918651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d732bbeb-0bab-48f3-91f4-4e2d122cf0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:14.926241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:01:15.195247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662091fc-d107-4af9-84b6-d15397652dab None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:15.196926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-662091fc-d107-4af9-84b6-d15397652dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:15.209965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662091fc-d107-4af9-84b6-d15397652dab None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:15.213391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:15.214012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:15.214290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:15.214590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:15.223263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:15.224670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a834b32-83cc-49de-be0a-1e1a6d47f52d service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:15.233643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 778/1539] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:49.265972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa36c1-90a7-4020-802e-4cd3629c3e7e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.266636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa36c1-90a7-4020-802e-4cd3629c3e7e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.266636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa36c1-90a7-4020-802e-4cd3629c3e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.296093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e1fa36c1-90a7-4020-802e-4cd3629c3e7e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.297060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae000-caec-4797-8d56-7e83c54dbead None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.297555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e1fa36c1-90a7-4020-802e-4cd3629c3e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.297655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae000-caec-4797-8d56-7e83c54dbead None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.297655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae000-caec-4797-8d56-7e83c54dbead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.315644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 762/1540] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.326297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-325ae000-caec-4797-8d56-7e83c54dbead None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.326899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.327192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.327589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.327711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-325ae000-caec-4797-8d56-7e83c54dbead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.341348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 779/1541] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:02:49.351963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705d3d-1067-4627-a422-2681d0b4af25 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.352304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705d3d-1067-4627-a422-2681d0b4af25 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.352528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705d3d-1067-4627-a422-2681d0b4af25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.359261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.360478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.378923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 763/1542] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.384838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.386874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-97705d3d-1067-4627-a422-2681d0b4af25 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.388323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-97705d3d-1067-4627-a422-2681d0b4af25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.394096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a62ab8-bf54-48fe-8621-c6e925d60a6d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.406583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 780/1543] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:02:49.407890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.408179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.408457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.411073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.411742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.412020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.412289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.413102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:49.413412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.415187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.431979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 764/1544] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:02:49.437937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.448450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.449680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.460198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 781/1545] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.461074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.467505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b15a-2c39-4f8d-8b9c-5a151a16cdb3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.473437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23cc6-0326-47ed-9e6b-43823e8714d1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.481710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.484958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.485637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.485920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.486213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.486928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.488889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.489167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30711955-4b63-4b41-8092-6f52bfad489b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.489287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.489459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.490283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:49.490620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.492727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691c2db-948f-4814-b386-41f466b28eb9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.515324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 765/1546] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:49.516666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b5ee-13d2-4a05-8b94-662f033d9b70 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.516951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b5ee-13d2-4a05-8b94-662f033d9b70 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.517231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b5ee-13d2-4a05-8b94-662f033d9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.518316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 782/1547] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:49.519659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389135-345d-4368-bb00-01a4752c2400 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.520180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389135-345d-4368-bb00-01a4752c2400 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.520468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389135-345d-4368-bb00-01a4752c2400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.536975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a9f6b5ee-13d2-4a05-8b94-662f033d9b70 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.538153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a9f6b5ee-13d2-4a05-8b94-662f033d9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.538282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-af389135-345d-4368-bb00-01a4752c2400 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.539353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-af389135-345d-4368-bb00-01a4752c2400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.593440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 766/1548] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.595015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae250fa-4079-4871-8589-81c329ca8f02 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.598766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 783/1549] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.598899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae250fa-4079-4871-8589-81c329ca8f02 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.598899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae250fa-4079-4871-8589-81c329ca8f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.599093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b8a80-8eb8-4760-b57e-4c9c2d0e111a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.599093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b8a80-8eb8-4760-b57e-4c9c2d0e111a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.599093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b8a80-8eb8-4760-b57e-4c9c2d0e111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.615118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cae250fa-4079-4871-8589-81c329ca8f02 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.615788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f40b8a80-8eb8-4760-b57e-4c9c2d0e111a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.616291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cae250fa-4079-4871-8589-81c329ca8f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.616925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f40b8a80-8eb8-4760-b57e-4c9c2d0e111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.624202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 767/1550] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.625482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.625702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 784/1551] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.625803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.626087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.627424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.627713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.627989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.648302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.648536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.649510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.649618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.659125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 768/1552] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.659901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.670419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ab263-5edb-42b5-bca9-637e32da3173 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.670948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 785/1553] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.671728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.676369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.677252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.677533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.677815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.678540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.679561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 15:02:49.679561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 15:02:49.679561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 15:02:49.680783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.681561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90e2b2-66c9-4862-89d8-360d38b3ca8e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.685996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.686678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.686956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.687227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.687901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.688890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 15:02:49.688890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 15:02:49.688890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 15:02:49.690059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f485129-e78d-42c8-82a7-c58bcdc1e97c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.707737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 769/1554] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:49.708557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.709928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 786/1555] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/users => generated 285 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:49.711178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378e950-9ddc-4bf5-b4d7-4d3561dd4f13 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.711452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378e950-9ddc-4bf5-b4d7-4d3561dd4f13 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.711726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378e950-9ddc-4bf5-b4d7-4d3561dd4f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.731057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5378e950-9ddc-4bf5-b4d7-4d3561dd4f13 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.732374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5378e950-9ddc-4bf5-b4d7-4d3561dd4f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.777614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4b5d5-c1bc-4116-b045-179a6d9c7b8a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.782216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.782911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.783196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.783473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.784185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:49.784489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.786077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d98-b0f6-4665-bcf7-d08617ccf7e0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.791265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 787/1556] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.792520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef52c-b79d-4ca7-abe3-acb3354a706a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.792812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef52c-b79d-4ca7-abe3-acb3354a706a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.793089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef52c-b79d-4ca7-abe3-acb3354a706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.798141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 770/1557] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:49.799274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5341ec9-26d1-4b18-a4d0-bc148be504cd None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.799562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5341ec9-26d1-4b18-a4d0-bc148be504cd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.799835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5341ec9-26d1-4b18-a4d0-bc148be504cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.810945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2e6ef52c-b79d-4ca7-abe3-acb3354a706a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.812138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2e6ef52c-b79d-4ca7-abe3-acb3354a706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.817055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f5341ec9-26d1-4b18-a4d0-bc148be504cd None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.818236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5341ec9-26d1-4b18-a4d0-bc148be504cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.819942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 788/1558] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.821488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.822474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.822750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.828609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 771/1559] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.829931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.830217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.830525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.842228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.843590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.847703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:49.848875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:49.851376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 789/1560] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.852151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.857075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 772/1561] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:49.857897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.864749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bf2d4-9ca8-41f9-9a11-19de564cb8fa None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.866711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cd6df-6d90-47cf-9c39-eaccfc71dd4d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.869903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.870604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.871128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.871400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.871923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.872107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.872559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.872848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.873122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.873731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.873836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.875714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.883514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 790/1562] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:49.884917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.894841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87dcc6-b9fa-409e-aee6-fb01cd14cb1d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.896609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 773/1563] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/users => generated 297 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:49.897385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.901513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.902263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.902707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.903077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.903889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:49.904318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.905323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 15:02:49.905323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 15:02:49.905323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 15:02:49.906205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.907047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95321366-47f4-4966-8ff2-3eb56505727c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.911664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.912317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.912602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.912876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.914060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.916040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.922956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 791/1564] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:49.923929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.933970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8e710-9241-4e8f-a7ed-6891af1beadf admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:49.939163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:49.943509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:49.944740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:49.944740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:49.944947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:49.945076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:49.947847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:49.948122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 774/1565] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:49 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:49.948953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.963048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 792/1566] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:49.963805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:49.973481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0a3d-ceed-4b36-9939-c269112778ab admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.007615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721947-5668-4d10-a587-5f58dcbb6f57 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.012642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.013298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.013575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.013847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.014603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:50.014905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.016491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.027367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.027719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 775/1567] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:50.028025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.028308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.028584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.028701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.029409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:50.029719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.031350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.037917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb239-4ec7-4efd-9659-0264a9ebf647 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.043138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.043797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.044076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.044345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.045051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:02:50.045352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.051185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 793/1568] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:50.051185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.051405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.066638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 776/1569] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:02:50.066638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.066925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a70a-61dc-4e56-a78d-aa30e6fe78e2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.067313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.067963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.068244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.068519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.069266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.069465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6f2c0-fed6-4e78-b837-0dfb6383e9d0 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.070873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.074909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.075601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.075881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.076160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.076867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.078499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.081077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 794/1570] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.081846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.091175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4e8c-fe4b-4902-8e06-ea22981f4e6e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.092174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 777/1571] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.093033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.095714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.096341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.096625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.096900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.097582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.099482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.102215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741588a-6ba2-42db-b3e3-8868b6a4d722 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.107199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.107820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.108096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.108369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.109052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.110958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.130527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 795/1572] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.130527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.134194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 778/1573] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.134949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.144021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f199-4839-48e5-8b69-e5f88cb8ba42 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.189098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ececc-1995-4dec-9521-dd3134790e80 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.193574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.194219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.194528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.194801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.195479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.197521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.199393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.200045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.200333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.200604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.201280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.203200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.222124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 796/1574] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.222949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.231914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 779/1575] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.231914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.287016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc705-e2ad-44ca-94a7-d8866113acfe admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.297437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2e4d-a22c-4ff8-8798-8dfce4b59dc3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.302339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.303110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.303394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.303667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.304353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.306868 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.330635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 780/1576] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.331676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.344095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.344773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.345051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.345321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.345995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.347918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.373858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 797/1577] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.375425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.390840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bddbc-dbba-4756-9b1f-d59a40fc0840 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.432887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27115-bf2a-474c-ae9a-bc949061b3c1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.437841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.438188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.438486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.438756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.439464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.441030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.444129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.445018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.445311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.445570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] PATH_INFO: `/v3/projects/c6d8e0508a22485787ab40235461c343/users/abf867ceaecc41be8dfc029c3a1938cb/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.451727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 798/1578] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.452522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.461133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d8e0508a22485787ab40235461c343, user_id=abf867ceaecc41be8dfc029c3a1938cb, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.461901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5db1d-5f27-446e-b64e-993ed66d3c6f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.462802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 15:02:50.462802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 15:02:50.462802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 15:02:50.464744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.466912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.467546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.467826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.468097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.468789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.470664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.477553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 781/1579] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:50 2022] PUT /identity/v3/projects/c6d8e0508a22485787ab40235461c343/users/abf867ceaecc41be8dfc029c3a1938cb/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:50.479580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.496652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 799/1580] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.497443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.506655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1c78-2753-4853-a34a-d1b1aec37a80 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.539287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235ff26-ae98-4a84-8e1e-aa72d3e99449 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.544340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.545002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.545285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.545561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.546271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.547888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.557279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 782/1581] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.558631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.564931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.565599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.565882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.566182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.566903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.567538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4293cb-455f-4c90-98b0-b0a96f23e985 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.568801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.572282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.572909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.573185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.573454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.574115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.576048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.589018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 800/1582] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.591283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.600023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 783/1583] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.600023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.604878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd7d1-13ba-4995-8dd2-f817399af053 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.607762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57167665-e3ef-4123-b1bf-1874dceebb59 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.609713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.610542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.610834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.611621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.612219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.612352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.613040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.613341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.613638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.614251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.614362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.616317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c22b9-57b4-40cb-840b-907d62f31575 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.636602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 801/1584] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.637461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.638303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 784/1585] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/users => generated 291 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.639488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.640001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.640512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.651193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb260d83-2b49-4522-a4ae-20fab3865f0f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.655942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.656809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.657089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.657362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] PATH_INFO: `/v3/projects/d851fe67b7e940f9b20979bdd619bf48/users/e690e810cdb54431af33651d13272239/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.668575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] MFA Rules not processed for user `abf867ceaecc41be8dfc029c3a1938cb`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:50.669806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:50.672029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d851fe67b7e940f9b20979bdd619bf48, user_id=e690e810cdb54431af33651d13272239, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.674850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.689469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 802/1586] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:50 2022] PUT /identity/v3/projects/d851fe67b7e940f9b20979bdd619bf48/users/e690e810cdb54431af33651d13272239/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:50.690546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.736360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 785/1587] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:50.737192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.747030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392ba4-e0fe-4cdf-8456-a43feb62b766 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.751454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.752082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.752359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.752628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.753075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f03633-88a6-4a65-8495-7ff926a037e0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.753333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.754957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.757762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.758450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.758739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.759020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.759735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.761298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.764505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 786/1588] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.765098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.770939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 803/1589] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.771610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.774349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272512b-cc4d-4831-ac8f-2a0f8963819a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.779553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.780220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.780534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.780814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.781107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ec6ca-106c-4678-8598-404bb35ddb41 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.781466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.783062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.785582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.786230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.786547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.786824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.787525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.789062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.793201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 787/1590] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.793918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.802064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 804/1591] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.802064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.807163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0f2c-52da-4097-9c6b-0548cd358238 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.809006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6ef04-5264-49a3-8044-3f56f93be8e1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.811698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.812342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.812623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.812897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.813438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.813596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.814057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.814338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.814641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.815268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c034df2-bb4f-4e85-aa12-c19eb863e8e4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.815384 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.816848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.825913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 788/1592] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.826998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.829829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.829967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 805/1593] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.830065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.830163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.838413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599f02-be54-4d20-89e9-1b3b5dde6c1f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.842950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.843574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.843864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.844141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.844817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.844967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:50.846722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.847202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:50.854962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 789/1594] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:50.856289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.866623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2665b38-a297-49fe-9d73-01e0ae4f87ef None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.871511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.875184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 806/1595] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:50 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:50.875322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.875322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.875601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.875691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] PATH_INFO: `/v3/projects/09c6a716650343c18e62bbe7235e48c8/users/aa56c83ab8494901b85d00d4f592e5ce/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.888036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c6a716650343c18e62bbe7235e48c8, user_id=aa56c83ab8494901b85d00d4f592e5ce, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.890762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.904766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 790/1596] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:50 2022] PUT /identity/v3/projects/09c6a716650343c18e62bbe7235e48c8/users/aa56c83ab8494901b85d00d4f592e5ce/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:50.905843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.937954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc9c0-48ca-495e-b036-0eae99c4b69d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.966796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc881a-f418-4d48-823d-782c159dbd92 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:50.971874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.972495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.972779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.973053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.973773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.975432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.984903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 791/1597] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:50.988348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:50.990900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:50.991567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:50.991853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:50.992129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:50.992842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:50.994469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:50.995166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3777-9bc4-4a8b-9a06-4f26a591bf5a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.000138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.000855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.001191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.001473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.002183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.003778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.007787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 807/1598] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:51.007787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.017423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 792/1599] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:51.017423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.017681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3278093-45e7-4391-ab06-a51c566e0eba admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.020589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.021237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.021524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.021832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.022582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.024160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.025946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cc116-b377-4231-a18b-4231fad49fde admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.030571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.031225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.031512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.031805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.032503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.034080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.034750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 808/1600] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:51.035833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.044587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 793/1601] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:51.048346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.057593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983775dc-d30e-4c3e-9232-70218062e885 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.062198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.062911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.063198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.063483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.064159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.066051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b9756-ff04-4322-b98c-76c8229e5516 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.088155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 794/1602] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:51.089408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.089695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.089976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.090221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ae7f-c096-4cfb-860b-89b799937dcf admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.094153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.094829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.095116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.095394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.104533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.106876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.108989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.110197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.115463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 809/1603] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:51.116291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.121293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 795/1604] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.121293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.127870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14348b3e-a5cc-4531-8e84-b85fdf2bfd00 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.128638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c403bc2-f103-4cb4-8dce-9cd1954b7c47 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.133286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.133665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.134844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.135450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.135589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.135698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] PATH_INFO: `/v3/projects/8ec07df1f4694dbd98691ceb14809277/users/12b3dbd4c078489db46a027b1e50c33d/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.135799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.136029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] PATH_INFO: `/v3/projects/0612cad27c8e4f77ad14bd9d10eba2b6/users/03c9259466f04438bad3ac99cc2af86b/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.152142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec07df1f4694dbd98691ceb14809277, user_id=12b3dbd4c078489db46a027b1e50c33d, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.154881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.155277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0612cad27c8e4f77ad14bd9d10eba2b6, user_id=03c9259466f04438bad3ac99cc2af86b, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.158188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.168746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 810/1605] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/8ec07df1f4694dbd98691ceb14809277/users/12b3dbd4c078489db46a027b1e50c33d/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.169540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.172316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 796/1606] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/0612cad27c8e4f77ad14bd9d10eba2b6/users/03c9259466f04438bad3ac99cc2af86b/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.182982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.234651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f2d70-c65f-4820-bf7e-2b347fbf3e9b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.254646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb56c-3e76-4a53-bc75-535b9c415a7e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.260164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.261287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.261439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.261716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] PATH_INFO: `/v3/projects/18cd09ed2b254bc798526f1c5eead93e/users/d60f544f7d9141aea7884338d7a901fb/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.277137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cd09ed2b254bc798526f1c5eead93e, user_id=d60f544f7d9141aea7884338d7a901fb, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.279975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.298442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 797/1607] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/18cd09ed2b254bc798526f1c5eead93e/users/d60f544f7d9141aea7884338d7a901fb/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.298442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.300065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.301068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.301438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.301802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] PATH_INFO: `/v3/projects/28efa99d77e4438ab681323ffb37d41e/users/80ced7317f0f4652a51bad39252805b2/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.323077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=28efa99d77e4438ab681323ffb37d41e, user_id=80ced7317f0f4652a51bad39252805b2, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.326224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.341306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 811/1608] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/28efa99d77e4438ab681323ffb37d41e/users/80ced7317f0f4652a51bad39252805b2/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.345398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.382809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866f92d-f0c1-4ac4-9b08-a953813d8f2f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.409296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5590b-327b-45be-ba64-19d0fd6c4bfb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.414831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.415530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.415819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.416094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.416807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.418938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.432680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 812/1609] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:51.432680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.432680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.446043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.447695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.448080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.448455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] PATH_INFO: `/v3/projects/8961e0a1d6d443a0847533b83aa1ea8b/users/06701ca6a5e0427c9d5b540133c522a0/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.463485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8961e0a1d6d443a0847533b83aa1ea8b, user_id=06701ca6a5e0427c9d5b540133c522a0, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.466910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222ecd-41d9-46e1-8389-fd8e4d8cceea admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.495138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726bfb1-5a58-443e-abd4-10d14a6b6dad admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.498525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 798/1610] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/8961e0a1d6d443a0847533b83aa1ea8b/users/06701ca6a5e0427c9d5b540133c522a0/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.501331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386d012-c2ab-4d3d-ab46-99c9fb5376fa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.502246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386d012-c2ab-4d3d-ab46-99c9fb5376fa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.504485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386d012-c2ab-4d3d-ab46-99c9fb5376fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.542563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4386d012-c2ab-4d3d-ab46-99c9fb5376fa None None] MFA Rules not processed for user `12b3dbd4c078489db46a027b1e50c33d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.543982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4386d012-c2ab-4d3d-ab46-99c9fb5376fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.564673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.565378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.565658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.565929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.615021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 799/1611] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.620504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2cd30-f6f0-4ab4-a59c-741a5a889b6d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.620504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2cd30-f6f0-4ab4-a59c-741a5a889b6d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.620504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2cd30-f6f0-4ab4-a59c-741a5a889b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.636176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.637645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f675c-2e38-4eab-b4bd-112d4a1ebd87 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.648927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 813/1612] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:51.651215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d1dc-77fb-4d16-87e5-afaa7c05ed83 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.651692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d1dc-77fb-4d16-87e5-afaa7c05ed83 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.652134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0d1dc-77fb-4d16-87e5-afaa7c05ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.654763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-18f2cd30-f6f0-4ab4-a59c-741a5a889b6d None None] MFA Rules not processed for user `03c9259466f04438bad3ac99cc2af86b`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.656100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-18f2cd30-f6f0-4ab4-a59c-741a5a889b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.674152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ada0d1dc-77fb-4d16-87e5-afaa7c05ed83 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.675970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ada0d1dc-77fb-4d16-87e5-afaa7c05ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.686433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 814/1613] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.687788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.688338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.688620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.709807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.711532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.723213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 815/1614] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.724484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.736442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20918740-c4f1-4ba6-83bf-1373a3daaea0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.742865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 800/1615] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.744775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.745726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.747187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.747187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.748792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.749475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.750124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] PATH_INFO: `/v3/projects/e29039bdc06847a8916faddbb466691d/users/9d610e6a2a2a42e1aab04c0458e636aa/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.769520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29039bdc06847a8916faddbb466691d, user_id=9d610e6a2a2a42e1aab04c0458e636aa, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.772654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe97ae-d8eb-4d79-8e4b-5f350c590d19 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.773619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:51.784476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:51.788386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 816/1616] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:51 2022] PUT /identity/v3/projects/e29039bdc06847a8916faddbb466691d/users/9d610e6a2a2a42e1aab04c0458e636aa/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:51.790113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e4f9-28b7-42f0-b6b3-5a7c81285330 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.790997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e4f9-28b7-42f0-b6b3-5a7c81285330 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.791444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e4f9-28b7-42f0-b6b3-5a7c81285330 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.793209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 817/1617] 10.222.0.83 () {54 vars in 961 bytes} [Tue May 10 15:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:02:51.799717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e4f9-28b7-42f0-b6b3-5a7c81285330 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.871451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e4f9-28b7-42f0-b6b3-5a7c81285330 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.874501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 801/1618] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:51.874501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.877809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.879050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.879434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.879807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.893452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.896175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.898424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a1a5-bd70-402d-9f36-02c48d0f508c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.912831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.914146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 818/1619] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:51.915482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.918269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.918642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.918926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.920125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.922582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.929689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cccbaa-6da6-4b13-adb7-286f6b8c6679 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:51.934804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:51.935720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.936091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.936498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:51.948553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:51.953111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb426d-d2e4-4cab-94eb-c02956bb988b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:51.965162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 802/1620] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:51.965162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:51.982231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 819/1621] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:51.987071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:51.987432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:51.987718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.042869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:52.044901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:52.116819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4bb52-a7d5-4fc4-9bb1-8900eb143494 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.129371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.130175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.130508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.131761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.132582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.138642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.142442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 820/1622] 10.222.0.83 () {58 vars in 1058 bytes} [Tue May 10 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:52.143577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.164348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 803/1623] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:52.165520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.189881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a42e37-89bf-4e36-af83-a9c3de9a3bd7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.205755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.206747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.207019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.207318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.208003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.209928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.227495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880b114-de82-4150-9ff7-62d0648b27b7 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.235566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.236427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.236808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.237206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.248980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 804/1624] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:52 2022] POST /identity/v3/users => generated 292 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:52.249839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.255740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.257272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.273148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 821/1625] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:52.275231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.276392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0d77-da2b-4be3-8292-b822913b18fd admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.287310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72b41-706f-4878-96c7-d42e52512b4b service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.291676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.292349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.292635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.292913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.299200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.308189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.309413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.309413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] PATH_INFO: `/v3/projects/e29039bdc06847a8916faddbb466691d/users/9d610e6a2a2a42e1aab04c0458e636aa/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.315252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29039bdc06847a8916faddbb466691d, user_id=9d610e6a2a2a42e1aab04c0458e636aa, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.323419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.366955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 822/1626] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:52 2022] PUT /identity/v3/projects/e29039bdc06847a8916faddbb466691d/users/9d610e6a2a2a42e1aab04c0458e636aa/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:52.369795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.420522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.422486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa09e5-648a-4b4a-84dc-10e3c183d969 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.459681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 805/1627] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:52.461300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.464529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.464831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.539680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfcb24-682c-4016-ada4-35fbfef2255e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.541226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] MFA Rules not processed for user `9d610e6a2a2a42e1aab04c0458e636aa`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:52.542515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:52.552738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.556492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.556492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.557081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.557420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.561581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.580923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 823/1628] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:52.586327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.605745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88507fd3-4560-4275-b72d-fc69a3e3b96e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.620087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.621251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.621533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.621808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] PATH_INFO: `/v3/projects/5a30ce85d2504da9af7adada79464d3b/users/f41a0749a38f45ebacc134cfb64dfb19/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.651874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a30ce85d2504da9af7adada79464d3b, user_id=f41a0749a38f45ebacc134cfb64dfb19, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.659925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.684133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 824/1629] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:52 2022] PUT /identity/v3/projects/5a30ce85d2504da9af7adada79464d3b/users/f41a0749a38f45ebacc134cfb64dfb19/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:52.688256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.690951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 806/1630] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:52.703103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.829878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d710-5d1b-4543-8316-8ca6633a2ce5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.841469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23a438-c08c-48d3-942d-579a5fd16472 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.853433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.854133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.857118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.860912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.861689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.863335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272a3c7-20d6-4a71-b6a8-82e3deb6a59c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.891186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 825/1631] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:52.896968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4aba-9064-4d78-929b-a126e263736f None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.897300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4aba-9064-4d78-929b-a126e263736f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.897528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4aba-9064-4d78-929b-a126e263736f None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.904542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 826/1632] 10.222.0.83 () {54 vars in 912 bytes} [Tue May 10 15:02:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:02:52.909755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4aba-9064-4d78-929b-a126e263736f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:52.938273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4aba-9064-4d78-929b-a126e263736f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:52.951019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.951939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.952223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.954854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:52.956022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:52.956529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:52.957025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.957627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] PATH_INFO: `/v3/projects/5a30ce85d2504da9af7adada79464d3b/users/f41a0749a38f45ebacc134cfb64dfb19/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:52.960014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a30ce85d2504da9af7adada79464d3b, user_id=f41a0749a38f45ebacc134cfb64dfb19, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:52.965404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39682b24-bb2a-4399-a351-e09e4626c741 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:52.993084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 827/1633] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:52 2022] PUT /identity/v3/projects/5a30ce85d2504da9af7adada79464d3b/users/f41a0749a38f45ebacc134cfb64dfb19/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:53.011347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:53.011347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:53.011663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:53.057355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:53.058637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:53.113565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:53.115070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11300c2c-4291-44df-96ba-ce4dfa95afe5 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:53.158002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 807/1634] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:53.159730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:53.160025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:53.160317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:53.249943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] MFA Rules not processed for user `f41a0749a38f45ebacc134cfb64dfb19`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:53.253945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:53.261152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 828/1635] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:53.262034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:53.378696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74009-d895-408f-bf65-302c21cc7783 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:53.383343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:53.385118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:53.385847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:53.386218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:53.399217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 808/1636] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:53.401108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:53.417913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2a885-2ab8-45e9-b653-25b2530b1511 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:53.423221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:53.425377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:53.425826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:53.426282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:53.468936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:53.471791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dc9cb-7ac1-44a9-8f93-2699ad9475d7 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:53.478894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:53.481057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:53.494550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 829/1638] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:53.515610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 809/1638] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:54.287498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:54.296688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdb4d2-3686-4e36-bb19-d8ad850fb44b service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:54.306364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:54.307072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:54.307350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:54.307628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:54.322625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:54.331214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:54.346451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 830/1639] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:02:54.576365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:54.576854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:54.576948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:54.710617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:54.722866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d24f3-bb9d-44aa-8048-f05a26fd1a1e service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:54.736185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:54.744595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:54.744882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:54.745570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:54.972154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:54.973911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545e4e3-3c45-49ea-9027-b1f69f50ad87 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:55.006587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 831/1640] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:55.060062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] MFA Rules not processed for user `80ced7317f0f4652a51bad39252805b2`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:55.061322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:55.221663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 810/1641] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 646 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:55.238059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6a0c-5fb1-416c-8fd3-974d6ab44537 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.238724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6a0c-5fb1-416c-8fd3-974d6ab44537 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.239595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6a0c-5fb1-416c-8fd3-974d6ab44537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.261564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-aefc6a0c-5fb1-416c-8fd3-974d6ab44537 None None] MFA Rules not processed for user `80ced7317f0f4652a51bad39252805b2`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:55.263118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-aefc6a0c-5fb1-416c-8fd3-974d6ab44537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:55.274280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 832/1642] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:55.286946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:55.313974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661541d-904b-4a94-9e05-aa260af34cad None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:55.330608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:55.332035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.332332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.333801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.355273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:55.355844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5122dc2-a5a7-4d7a-98d0-ff331190e00c None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.356199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5122dc2-a5a7-4d7a-98d0-ff331190e00c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.356547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5122dc2-a5a7-4d7a-98d0-ff331190e00c None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.356682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6ed75-198a-414c-b3ed-4babae55087a service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:55.358172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 833/1643] 10.222.0.83 () {54 vars in 961 bytes} [Tue May 10 15:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 15:02:55.368137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d35b5-c789-48a4-878c-18dc47b85736 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.368531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d35b5-c789-48a4-878c-18dc47b85736 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.368879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d35b5-c789-48a4-878c-18dc47b85736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.382550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 811/1644] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:55.442999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2e7d35b5-c789-48a4-878c-18dc47b85736 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:55.450329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2e7d35b5-c789-48a4-878c-18dc47b85736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:55.490810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 834/1645] 10.222.0.83 () {58 vars in 1058 bytes} [Tue May 10 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:55.496238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0ac26-5b0b-4532-9f6f-1357893b793b None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.496948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0ac26-5b0b-4532-9f6f-1357893b793b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.497380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0ac26-5b0b-4532-9f6f-1357893b793b None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.500059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 812/1646] 10.222.0.83 () {54 vars in 912 bytes} [Tue May 10 15:02:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:02:55.505312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690dc7-53de-4a3b-9c58-2992d045050a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.505739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690dc7-53de-4a3b-9c58-2992d045050a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.506200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690dc7-53de-4a3b-9c58-2992d045050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.556284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fd690dc7-53de-4a3b-9c58-2992d045050a None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:55.561038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fd690dc7-53de-4a3b-9c58-2992d045050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:55.593601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 835/1647] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:55.607484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0ac26-5b0b-4532-9f6f-1357893b793b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:55.626309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0ac26-5b0b-4532-9f6f-1357893b793b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:55.639244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:55.641063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.641331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.641595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.664589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:55.666422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:55.702584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 813/1648] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:55.823956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:55.824441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:55.824629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:55.894710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] MFA Rules not processed for user `d60f544f7d9141aea7884338d7a901fb`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:55.896425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:56.084489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 836/1649] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:02:56.090699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:56.277821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef00f-b977-4ae7-a6bd-403d0bbbc85f service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:56.291653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:56.292616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:56.292762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:56.293047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:56.293749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:56.295892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:56.357784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 814/1650] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:02:56 2022] POST /identity/v3/projects => generated 403 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:02:56.361852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:56.377768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e2842-7484-46b2-a1aa-ef58bd849ddc None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:56.598762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:56.601674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:56.602707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:56.605505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:56.606592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:56.610108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:56.661677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 837/1651] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:02:56 2022] POST /identity/v3/users => generated 308 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:02:56.665891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:56.683060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5d0f-1e59-4a08-a438-d220b186cbb3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:56.697026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:56.701412 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:56.702199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:56.703405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:56.705817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:56.708365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:56.726656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 815/1652] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:56.733876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:56.748973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce142d-db59-4db2-8bcb-e0dc10a54723 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:56.755432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:56.756402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:56.756688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:56.756961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] PATH_INFO: `/v3/projects/c16851676c6b4b81a8c64bf171e1768f/users/e478bf14fbdb408b88213bec0481447e/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:56.794426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16851676c6b4b81a8c64bf171e1768f, user_id=e478bf14fbdb408b88213bec0481447e, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:56.800122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290622-ecf2-44f7-adde-6ab98555bf31 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:56.822438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 838/1653] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:56 2022] PUT /identity/v3/projects/c16851676c6b4b81a8c64bf171e1768f/users/e478bf14fbdb408b88213bec0481447e/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:02:56.824911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:56.840696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b337c2-5110-41d2-93ec-1975d8073d0b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:56.987512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efda5a-7398-4897-9fbe-cc79a39386a8 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:56.988401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efda5a-7398-4897-9fbe-cc79a39386a8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:56.988885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efda5a-7398-4897-9fbe-cc79a39386a8 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:56.992298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 839/1654] 10.222.0.83 () {54 vars in 912 bytes} [Tue May 10 15:02:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 15:02:57.003770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:57.006267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.006679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.008533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.009618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:57.010582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.011003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.011247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.013108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:57.042852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 816/1655] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:02:57.043763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:57.058691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19a201-7699-4af0-9118-2906a1df51e7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:57.065497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:57.066571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.067257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.067575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] PATH_INFO: `/v3/projects/c16851676c6b4b81a8c64bf171e1768f/users/e478bf14fbdb408b88213bec0481447e/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.091846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:57.093671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16851676c6b4b81a8c64bf171e1768f, user_id=e478bf14fbdb408b88213bec0481447e, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:57.093829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:57.105613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:57.156823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 817/1656] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:02:57 2022] PUT /identity/v3/projects/c16851676c6b4b81a8c64bf171e1768f/users/e478bf14fbdb408b88213bec0481447e/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:57.218625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 840/1657] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:02:57.242497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:57.408190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:57.418106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f841-87b3-405b-8566-e56cf152f449 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:57.422898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47591ed3-4c84-4ddd-850c-fa4e63b2b1e6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:57.432304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:57.433001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.433336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.433550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.455243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:57.456559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cf888-918d-4ad7-b61a-5cf81dc99cdf service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:57.497866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 818/1658] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:57.611469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:57.612970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.613458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.614126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] PATH_INFO: `/v3/users/12b3dbd4c078489db46a027b1e50c33d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.616681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b3dbd4c078489db46a027b1e50c33d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:57.620491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:57.720894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b3dbd4c078489db46a027b1e50c33d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:57.722764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b3dbd4c078489db46a027b1e50c33d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:57.737549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b3dbd4c078489db46a027b1e50c33d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:57.739626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b3dbd4c078489db46a027b1e50c33d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:57.780453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b3dbd4c078489db46a027b1e50c33d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:57.844938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 841/1659] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:02:57 2022] DELETE /identity/v3/users/12b3dbd4c078489db46a027b1e50c33d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:02:57.885343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:57.885343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:57.885343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:57.911561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:57.912802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:58.077403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 819/1660] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:02:58.088776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:58.091454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:58.219031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf3e55-02a4-4375-9a07-d2632e9dcc31 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:58.223221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:58.224117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:58.224538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:58.224976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:58.239900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:58.241719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e20c2-ff77-4887-bf23-9078bb96d9bb service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:58.264269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 842/1661] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:58.865619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:58.867235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:58.926651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5ecce-fbc4-4e66-a1e1-9399b51807d7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:58.953384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:58.956561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:58.965017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:58.966503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] PATH_INFO: `/v3/users/abf867ceaecc41be8dfc029c3a1938cb` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:58.968256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf867ceaecc41be8dfc029c3a1938cb)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:58.971707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: WARNING py.warnings [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 15:02:58.971707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: warnings.warn(msg) May 10 15:02:58.971707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: May 10 15:02:58.973330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:59.128025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf867ceaecc41be8dfc029c3a1938cb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:59.153042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf867ceaecc41be8dfc029c3a1938cb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:59.153969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf867ceaecc41be8dfc029c3a1938cb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:59.155006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf867ceaecc41be8dfc029c3a1938cb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:59.172908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9034849b-14ee-4102-a5a8-69746b09fbdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf867ceaecc41be8dfc029c3a1938cb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:02:59.174185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a60c-7010-41a2-8ac5-33864be7b584 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:59.175454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a60c-7010-41a2-8ac5-33864be7b584 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:59.176210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a60c-7010-41a2-8ac5-33864be7b584 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:59.178166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 843/1662] 10.222.0.83 () {54 vars in 960 bytes} [Tue May 10 15:02:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:02:59.187670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:59.188184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:59.188698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:59.265746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 820/1663] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:02:58 2022] DELETE /identity/v3/users/abf867ceaecc41be8dfc029c3a1938cb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:02:59.267336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:59.267626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:59.270180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:59.270691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] MFA Rules not processed for user `01c876294f0a400f8bcd833377df454c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:59.271983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:59.294681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:02:59.296125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:59.454203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 821/1664] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:02:59.479460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 844/1665] 10.222.0.83 () {58 vars in 1057 bytes} [Tue May 10 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 10 15:02:59.489756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:59.505792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:02:59.511870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:59.512785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baf4cc-e54a-451b-9779-035235bf255d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:59.530823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:59.531760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:59.532056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:59.532334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:59.558531 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:02:59.573887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:59.576042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff39ec-945c-4015-befa-6a0e6f281771 service placement] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:59.606141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 822/1666] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:02:59.723400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd56aa3-2934-4e8d-a60a-06cb19ed1ba1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:02:59.739257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:02:59.744151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:02:59.752471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:02:59.754852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:02:59.817580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:02:59.820006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:02:59.893967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 845/1667] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:00.937851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1a23-e2d8-4082-aa27-62f7fdbad017 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:00.938431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1a23-e2d8-4082-aa27-62f7fdbad017 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:00.938431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1a23-e2d8-4082-aa27-62f7fdbad017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.009470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.011930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.041878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbf4b6-b18c-4350-9868-9049468bacdf service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.046145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-45ef1a23-e2d8-4082-aa27-62f7fdbad017 None None] MFA Rules not processed for user `e690e810cdb54431af33651d13272239`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:01.050000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-45ef1a23-e2d8-4082-aa27-62f7fdbad017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.053631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.055111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.055409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.056557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] PATH_INFO: `/v3/projects/8ec07df1f4694dbd98691ceb14809277` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.058244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec07df1f4694dbd98691ceb14809277)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.059762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: WARNING py.warnings [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 15:03:01.059762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: warnings.warn(msg) May 10 15:03:01.059762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: May 10 15:03:01.062641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.170681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec07df1f4694dbd98691ceb14809277', 'request_id': 'req-33b86d42-575f-4281-954b-02e5d802e7f7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:01.191376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec07df1f4694dbd98691ceb14809277', 'request_id': 'req-33b86d42-575f-4281-954b-02e5d802e7f7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:01.199182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec07df1f4694dbd98691ceb14809277', 'request_id': 'req-33b86d42-575f-4281-954b-02e5d802e7f7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:01.220398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 823/1668] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:01.227445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] The token cache is being invalidate because project 8ec07df1f4694dbd98691ceb14809277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:01.228019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:01.230875 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.231170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.231453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.234039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33b86d42-575f-4281-954b-02e5d802e7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:01.243638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 846/1669] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:01 2022] DELETE /identity/v3/projects/8ec07df1f4694dbd98691ceb14809277 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:01.245131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7525-c5b6-4632-8a53-1ea4bf380913 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.245425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7525-c5b6-4632-8a53-1ea4bf380913 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.246291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7525-c5b6-4632-8a53-1ea4bf380913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.264996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] MFA Rules not processed for user `e690e810cdb54431af33651d13272239`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:01.271277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.281181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e59b7525-c5b6-4632-8a53-1ea4bf380913 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:01.282687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e59b7525-c5b6-4632-8a53-1ea4bf380913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.365176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 847/1670] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:01.366831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.367121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.367400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.375628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 824/1671] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:01.384392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.386518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.411533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:01.412773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.438970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 848/1672] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:01.440278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.453490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398117d-d6ae-42e9-97e6-e18350ba9407 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.469205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.469892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.470182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.470694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.471676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:01.471759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.474143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.497772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 849/1673] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:03:01.499949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.511251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58bdd-e1be-4257-9898-47fecb8fffd0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.517501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.518222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.518333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba02f-4223-4a48-a042-5f1397f5302d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.518543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.518827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.519693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.521751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.530814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.531779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.532152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.532737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.536378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:01.555304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.558309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.570235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 825/1674] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:01.583243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 850/1675] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:01.588990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.606718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae7a8e-ba26-41ac-9534-43eb6e54dda4 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.705788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.706813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.707162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.707506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.708478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.710711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.735796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 826/1676] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:01.739479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.750446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd850ce-5feb-4883-8749-958d763b110e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.755434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.756336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.756441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.756787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.757669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.759419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.777273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 851/1677] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:01.783984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.808964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38785-8a2f-4521-b047-bf48209ca267 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.831230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.832321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.832816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.833338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.834279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.836170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824079-65fe-4e51-9a35-225b88630118 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.860454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 827/1678] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:01.864858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:01.886041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906df96-7cd1-458b-96cb-12b2378533fc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:01.890934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:01.891841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.892123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.892395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] PATH_INFO: `/v3/projects/72cd60cf3ecb4a448651d24b92fbe567/users/6310ff859900402488c6fdc0f54d6aea/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.928988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cd60cf3ecb4a448651d24b92fbe567, user_id=6310ff859900402488c6fdc0f54d6aea, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:01.931761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:01.956960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 852/1679] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:01 2022] PUT /identity/v3/projects/72cd60cf3ecb4a448651d24b92fbe567/users/6310ff859900402488c6fdc0f54d6aea/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:01.969234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080010-3e46-4235-9518-27a7947e222e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:01.969744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080010-3e46-4235-9518-27a7947e222e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:01.969815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5080010-3e46-4235-9518-27a7947e222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:01.998943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f5080010-3e46-4235-9518-27a7947e222e None None] MFA Rules not processed for user `6310ff859900402488c6fdc0f54d6aea`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:02.000445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5080010-3e46-4235-9518-27a7947e222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:02.090181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 828/1680] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:02.102556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:02.104690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:02.177623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210d82-5e19-4f76-b456-a64d4e13edc5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.177623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210d82-5e19-4f76-b456-a64d4e13edc5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.177623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210d82-5e19-4f76-b456-a64d4e13edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.201766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424787-ef96-495c-af8f-f09f77f5cc72 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:02.208487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:02.209210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.210284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.210669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.223371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:02.225493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:02.246430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 853/1681] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:02.247649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:02.249303 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:02.309818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-be210d82-5e19-4f76-b456-a64d4e13edc5 None None] MFA Rules not processed for user `e478bf14fbdb408b88213bec0481447e`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:02.311483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-be210d82-5e19-4f76-b456-a64d4e13edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:02.470575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 829/1682] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:02.472042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221393f3-e9d5-4c40-bc51-69f3c37a9648 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.474188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221393f3-e9d5-4c40-bc51-69f3c37a9648 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.476667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221393f3-e9d5-4c40-bc51-69f3c37a9648 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.479572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 830/1683] 10.222.0.83 () {54 vars in 912 bytes} [Tue May 10 15:03:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:03:02.484066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.484468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.484848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.491703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4ed58-f000-4e42-8ee0-eae6005fedb9 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:02.498203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:02.501442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.501746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.502030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] PATH_INFO: `/v3/projects/c6d8e0508a22485787ab40235461c343` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.503652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d8e0508a22485787ab40235461c343)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:02.506877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:02.527512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:02.528854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:02.641740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 831/1684] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:02.647237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d8e0508a22485787ab40235461c343', 'request_id': 'req-25172aea-1b15-450d-932e-8e06549b9d1b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.655797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d8e0508a22485787ab40235461c343', 'request_id': 'req-25172aea-1b15-450d-932e-8e06549b9d1b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.664129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d8e0508a22485787ab40235461c343', 'request_id': 'req-25172aea-1b15-450d-932e-8e06549b9d1b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.701653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] The token cache is being invalidate because project c6d8e0508a22485787ab40235461c343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:02.702824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.703633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.705857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 854/1685] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:02 2022] DELETE /identity/v3/projects/c6d8e0508a22485787ab40235461c343 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:02.710041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:02.723546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53759-2bda-45a3-af0f-e34e98253b7e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:02.793778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:02.794666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:02.794949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:02.795442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] PATH_INFO: `/v3/users/9d610e6a2a2a42e1aab04c0458e636aa` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:02.796682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d610e6a2a2a42e1aab04c0458e636aa)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:02.799753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:02.879509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d610e6a2a2a42e1aab04c0458e636aa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.896665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d610e6a2a2a42e1aab04c0458e636aa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.898789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d610e6a2a2a42e1aab04c0458e636aa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.900380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d610e6a2a2a42e1aab04c0458e636aa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.933688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7c11d75c-783d-46be-9c36-b2af608980a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d610e6a2a2a42e1aab04c0458e636aa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:02.990523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 832/1686] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:02 2022] DELETE /identity/v3/users/9d610e6a2a2a42e1aab04c0458e636aa => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:03.062592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:03.064308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:03.166265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25172aea-1b15-450d-932e-8e06549b9d1b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:03.231162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:03.231770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:03.232451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:03.232993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:03.235385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:03.248964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:03.250729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:03.262460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 855/1687] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:04.123685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:04.124607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:04.125113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:04.175968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:04.181481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:04.331077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 833/1688] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:03:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:04.597854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:04.599361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:04.617390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:04.618934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:04.860402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a0a3-1262-44fa-9b5a-838fe7462883 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:04.861629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47166668-977c-4b79-87d8-0b4f1ea5e4b7 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:04.886664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:04.887659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:04.888160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:04.888629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] PATH_INFO: `/v3/projects/e29039bdc06847a8916faddbb466691d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:04.890997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29039bdc06847a8916faddbb466691d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:04.893025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:04.893701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:04.893984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:04.894262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:04.895914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:04.938781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:04.940594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79f5d-f083-4cdd-a750-2bdc3510ea4c service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:04.991294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 834/1689] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:03:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:03:05.005772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29039bdc06847a8916faddbb466691d', 'request_id': 'req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:05.022835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29039bdc06847a8916faddbb466691d', 'request_id': 'req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:05.039310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29039bdc06847a8916faddbb466691d', 'request_id': 'req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:05.074656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] The token cache is being invalidate because project e29039bdc06847a8916faddbb466691d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:05.075090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:05.078487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4f001cc7-b6b4-4ff0-be37-7b0af0e32c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:05.080959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 856/1690] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:04 2022] DELETE /identity/v3/projects/e29039bdc06847a8916faddbb466691d => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:05.086357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.086927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.087239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.124891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc0b3-de6f-47c4-85df-1758c01addb0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.125627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc0b3-de6f-47c4-85df-1758c01addb0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.126105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dc0b3-de6f-47c4-85df-1758c01addb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.130445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:05.132757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:05.188064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-257dc0b3-de6f-47c4-85df-1758c01addb0 None None] MFA Rules not processed for user `e690e810cdb54431af33651d13272239`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:05.193710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-257dc0b3-de6f-47c4-85df-1758c01addb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:05.406770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 857/1691] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:05.413423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 835/1692] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:05.421528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.422081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.422578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.475671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:05.478951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:05.509623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 858/1693] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:05.511723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:05.527717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361166a-5f5c-4cc3-b7a9-722072450364 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:05.551790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:05.556341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.558529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.560463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.563858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:05.564372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:05.569215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:05.629323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:05.629835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 836/1694] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:05.640615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4a427-3d5a-447f-8540-27d18e407604 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:05.671462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:05.676465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.677101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.678584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.681062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:05.687067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:05.777416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 859/1695] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:05 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:05.782210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:05.815756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600e1c-8f68-4929-9a4e-6d5319eab55c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:05.986349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:05.987366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:05.987792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:05.988233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:05.989361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:05.991960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:06.054348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 837/1696] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:05 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:06.055772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:06.093141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1dd9a-7183-4344-a51b-742a2017a48a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:06.133220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:06.138693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:06.139077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:06.139452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:06.140322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:06.142082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:06.184970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 860/1697] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:06.190021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:06.222162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42ed84-8aba-412b-b712-f8696952c229 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:06.247330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:06.252201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:06.257225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:06.259562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:06.261079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:06.272667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1dfbf-77be-4215-9b32-4be313ef9c74 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:06.312716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:06.312896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 838/1698] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:06.331164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71840d51-cd50-40be-b5de-ae6b4499246d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:06.351428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:06.353864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:06.371280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:06.371493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] PATH_INFO: `/v3/projects/ebce7aa6dc5f4a999ef3976426b01e51/users/dd09790500ff42dc96cf10e5782186a1/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:06.429277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebce7aa6dc5f4a999ef3976426b01e51, user_id=dd09790500ff42dc96cf10e5782186a1, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:06.432725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:06.472279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 861/1699] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:06 2022] PUT /identity/v3/projects/ebce7aa6dc5f4a999ef3976426b01e51/users/dd09790500ff42dc96cf10e5782186a1/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:06.483325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:06.483901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:06.484342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:06.625708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] MFA Rules not processed for user `dd09790500ff42dc96cf10e5782186a1`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:06.647953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:06.849326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:06.852154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 839/1700] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:07.149015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763f74-5a0e-4bcd-9e31-7fbad8377bd2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:07.167222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:07.171631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:07.172080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:07.172515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:07.173390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:07.177431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:07.318028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 862/1701] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:06 2022] POST /identity/v3/projects => generated 385 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:07.320596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:07.342254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9ad1c-6ca4-437d-b377-8b97be2e78e0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:07.615432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:07.616376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:07.616923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:07.617402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:07.618344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:07.630350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:07.669807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 840/1702] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:07 2022] POST /identity/v3/users => generated 299 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:07.673901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:07.693369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0577cd-7a07-4d45-9c66-e53a428ee311 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:07.717612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:07.718317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:07.718693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:07.718972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:07.719721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:07.721403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:07.771594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 863/1703] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:07.775831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:07.792882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079df11-9eb7-4e55-a6f6-3b104c56cc9d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:07.808456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:07.814459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:07.815185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:07.815931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] PATH_INFO: `/v3/projects/02291b541fec4cf68e8e6c7564abec07/users/e1fc3d3c70d94305bcb0b316db2dcf9b/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:07.865938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02291b541fec4cf68e8e6c7564abec07, user_id=e1fc3d3c70d94305bcb0b316db2dcf9b, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:07.869423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:07.897313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 841/1704] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:07 2022] PUT /identity/v3/projects/02291b541fec4cf68e8e6c7564abec07/users/e1fc3d3c70d94305bcb0b316db2dcf9b/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:07.901725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:07.922617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6126d95-f166-4e38-a286-09fafedd702c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:08.044802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:08.045713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:08.046222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:08.051080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:08.052552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:08.054354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e76df-e2c9-4ca1-bb27-83fb19b368f1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:08.067423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 864/1705] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:08.071717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:08.089264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcb47-efe2-4bf4-a6cd-8b807386473d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:08.101505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:08.105796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:08.106805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:08.107438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] PATH_INFO: `/v3/projects/02291b541fec4cf68e8e6c7564abec07/users/e1fc3d3c70d94305bcb0b316db2dcf9b/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:08.112716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02291b541fec4cf68e8e6c7564abec07, user_id=e1fc3d3c70d94305bcb0b316db2dcf9b, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:08.123674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:08.156975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 842/1706] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:08 2022] PUT /identity/v3/projects/02291b541fec4cf68e8e6c7564abec07/users/e1fc3d3c70d94305bcb0b316db2dcf9b/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:08.169659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa719d-4684-4474-8f5e-3e2ea7c4886b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:08.170032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa719d-4684-4474-8f5e-3e2ea7c4886b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:08.170431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa719d-4684-4474-8f5e-3e2ea7c4886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:08.227806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e6fa719d-4684-4474-8f5e-3e2ea7c4886b None None] MFA Rules not processed for user `e1fc3d3c70d94305bcb0b316db2dcf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:08.229154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e6fa719d-4684-4474-8f5e-3e2ea7c4886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:08.424851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:08.425535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 865/1707] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:08.432350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:08.569814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c83929-d854-417c-b2ed-e78fe7615575 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:08.574826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:08.576375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:08.576870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:08.577300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:08.598687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:08.600439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:08.632068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 843/1708] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:09.588766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:09.589463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:09.589877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:09.665528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:09.669966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:09.956212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 866/1709] 10.222.0.87 () {58 vars in 1015 bytes} [Tue May 10 15:03:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:10.392919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:10.394203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:10.443648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bfa04-8564-45aa-b377-1a39ee87cd53 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:10.464579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:10.468413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:10.468938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:10.471298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:10.488038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:10.510439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:10.517897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014e9c-92b3-465a-be8d-ed2b0ff91a8b service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:10.574960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 844/1710] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:10.728510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d2eb2-3637-4b2c-a5f1-12d3d06b9321 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:10.770743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:10.776891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:10.785418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:10.787338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] PATH_INFO: `/v3/users/dd09790500ff42dc96cf10e5782186a1` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:10.796656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd09790500ff42dc96cf10e5782186a1)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:10.798675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:11.009405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd09790500ff42dc96cf10e5782186a1'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:11.010846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd09790500ff42dc96cf10e5782186a1'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:11.045667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd09790500ff42dc96cf10e5782186a1'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:11.047653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd09790500ff42dc96cf10e5782186a1'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:11.070730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd09790500ff42dc96cf10e5782186a1'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:11.136864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 867/1711] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:10 2022] DELETE /identity/v3/users/dd09790500ff42dc96cf10e5782186a1 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:11.141924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7f12-152f-4712-a40e-bb98dc2ea927 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:11.142341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7f12-152f-4712-a40e-bb98dc2ea927 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:11.142807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7f12-152f-4712-a40e-bb98dc2ea927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:11.183838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8aed7f12-152f-4712-a40e-bb98dc2ea927 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:11.185077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8aed7f12-152f-4712-a40e-bb98dc2ea927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:11.419356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 845/1712] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:11.503954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:11.505773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:11.672084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336d443-dc30-4c2a-9d26-9576dd6e67ad admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:11.681657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:11.685783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:11.686299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:11.686730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:11.709744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:11.714611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:11.741310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 868/1713] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:12.643736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:12.644171 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:12.645092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:12.743154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] MFA Rules not processed for user `06701ca6a5e0427c9d5b540133c522a0`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:12.744401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:12.975154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 846/1714] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:13.002894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:13.005230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:13.265437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5141-f314-46f1-9b70-92a074054526 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:13.289545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:13.290973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:13.291315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:13.291599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:13.330863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:13.332846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:13.360464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 869/1715] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:14.591721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:14.593403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:14.629651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8184f-a607-44de-b414-a5b9d6a724f5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:14.643591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:14.644505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:14.644790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:14.645070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] PATH_INFO: `/v3/users/d60f544f7d9141aea7884338d7a901fb` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:14.653848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60f544f7d9141aea7884338d7a901fb)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:14.655882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:14.822524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60f544f7d9141aea7884338d7a901fb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:14.848707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60f544f7d9141aea7884338d7a901fb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:14.852435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60f544f7d9141aea7884338d7a901fb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:14.854590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60f544f7d9141aea7884338d7a901fb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:14.871615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60f544f7d9141aea7884338d7a901fb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:14.973467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 847/1716] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:14 2022] DELETE /identity/v3/users/d60f544f7d9141aea7884338d7a901fb => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:15.507361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:15.510893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:15.538361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b80e-1471-4c0d-a428-f4905a991325 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:15.776820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:15.781514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:15.781912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:15.782201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] PATH_INFO: `/v3/projects/ebce7aa6dc5f4a999ef3976426b01e51` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:15.785996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebce7aa6dc5f4a999ef3976426b01e51)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:15.794057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:15.937154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebce7aa6dc5f4a999ef3976426b01e51', 'request_id': 'req-136fa7c4-2736-4e48-a0c1-43c76604fff8'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:15.969627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebce7aa6dc5f4a999ef3976426b01e51', 'request_id': 'req-136fa7c4-2736-4e48-a0c1-43c76604fff8'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:15.984727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebce7aa6dc5f4a999ef3976426b01e51', 'request_id': 'req-136fa7c4-2736-4e48-a0c1-43c76604fff8'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.019976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] The token cache is being invalidate because project ebce7aa6dc5f4a999ef3976426b01e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:16.020642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.021568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-136fa7c4-2736-4e48-a0c1-43c76604fff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.026019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 870/1717] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:15 2022] DELETE /identity/v3/projects/ebce7aa6dc5f4a999ef3976426b01e51 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:16.030277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:16.033415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:16.098740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf1d0-8aed-4089-b41d-08ccaa448f2f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:16.238382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:16.238931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:16.239403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:16.241192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 871/1718] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:03:16.348258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:16.353681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:16.355578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:16.362457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:16.363577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:16.364553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] PATH_INFO: `/v3/users/e1fc3d3c70d94305bcb0b316db2dcf9b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:16.373930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fc3d3c70d94305bcb0b316db2dcf9b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:16.392744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:16.409992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b8e89-7cc8-4efd-a57b-a2b1dbcc6510 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:16.435290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:16.436793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:16.436793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:16.437231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] PATH_INFO: `/v3/users/6310ff859900402488c6fdc0f54d6aea` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:16.442972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6310ff859900402488c6fdc0f54d6aea)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:16.458017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:16.541467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fc3d3c70d94305bcb0b316db2dcf9b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.569393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fc3d3c70d94305bcb0b316db2dcf9b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.571854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fc3d3c70d94305bcb0b316db2dcf9b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.572771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fc3d3c70d94305bcb0b316db2dcf9b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.589324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-208c48fd-8eb6-489c-a943-351436bcccf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fc3d3c70d94305bcb0b316db2dcf9b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.594178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6310ff859900402488c6fdc0f54d6aea'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.597100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6310ff859900402488c6fdc0f54d6aea'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.620839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6310ff859900402488c6fdc0f54d6aea'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.638031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6310ff859900402488c6fdc0f54d6aea'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.646274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 848/1719] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:16 2022] DELETE /identity/v3/users/e1fc3d3c70d94305bcb0b316db2dcf9b => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:16.648768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:16.650001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:16.650556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:16.653686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-222e1175-73c3-42c8-8784-f473350424fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6310ff859900402488c6fdc0f54d6aea'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:16.696648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 872/1720] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:16 2022] DELETE /identity/v3/users/6310ff859900402488c6fdc0f54d6aea => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:16.698383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:16.698908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:16.700276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:16.703612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:16.705164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:16.723408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:16.731231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:16.906298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 873/1721] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:16.999885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 849/1722] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:03:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:17.009467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:17.011266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:17.206696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a0a8b-9041-4101-91cd-c92f07c2530a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:17.214320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:17.215744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:17.216169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:17.217208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:17.256708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:17.257370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:17.259018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:17.259505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:17.277074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 874/1723] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:17.286480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf31ae0-81bc-4d88-ad03-af38e2e24217 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:17.302197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:17.304026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:17.304710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:17.306095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:17.369742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:17.374325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:17.429206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 850/1724] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:17.453492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:17.454893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:17.518800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02985-2120-4126-9e9d-63f8ade2d658 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:17.535077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:17.536383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:17.536882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:17.537355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] PATH_INFO: `/v3/projects/18cd09ed2b254bc798526f1c5eead93e` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:17.541763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cd09ed2b254bc798526f1c5eead93e)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:17.544470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:17.728906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cd09ed2b254bc798526f1c5eead93e', 'request_id': 'req-a6549ccb-70c1-412b-b7c1-beb589c11d46'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:17.796658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cd09ed2b254bc798526f1c5eead93e', 'request_id': 'req-a6549ccb-70c1-412b-b7c1-beb589c11d46'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:17.830169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cd09ed2b254bc798526f1c5eead93e', 'request_id': 'req-a6549ccb-70c1-412b-b7c1-beb589c11d46'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:17.895766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] The token cache is being invalidate because project 18cd09ed2b254bc798526f1c5eead93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:17.895766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:17.904057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:17.912210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 875/1725] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:17 2022] DELETE /identity/v3/projects/18cd09ed2b254bc798526f1c5eead93e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:17.918843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:17.945278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905435de-b8f3-4f22-a4b3-c65025fb3859 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:18.041756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:18.042751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:18.043026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:18.043472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] PATH_INFO: `/v3/users/e478bf14fbdb408b88213bec0481447e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:18.046587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e478bf14fbdb408b88213bec0481447e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:18.049155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:18.231636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e478bf14fbdb408b88213bec0481447e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.258138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e478bf14fbdb408b88213bec0481447e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.261660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e478bf14fbdb408b88213bec0481447e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.266335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e478bf14fbdb408b88213bec0481447e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.278569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3f12b29a-6750-4764-90a6-164d29ba365e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e478bf14fbdb408b88213bec0481447e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.357211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 851/1726] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:17 2022] DELETE /identity/v3/users/e478bf14fbdb408b88213bec0481447e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:18.526371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:18.537845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:18.577493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549ccb-70c1-412b-b7c1-beb589c11d46 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:18.809540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:18.812032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:18.813402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:18.814847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] PATH_INFO: `/v3/projects/02291b541fec4cf68e8e6c7564abec07` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:18.822519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] RBAC: Authorizing `identity:delete_project(project_id=02291b541fec4cf68e8e6c7564abec07)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:18.825176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:18.968376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02291b541fec4cf68e8e6c7564abec07', 'request_id': 'req-be643d2d-1a33-40a1-8080-667ed6ab638d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.990869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02291b541fec4cf68e8e6c7564abec07', 'request_id': 'req-be643d2d-1a33-40a1-8080-667ed6ab638d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:18.999054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02291b541fec4cf68e8e6c7564abec07', 'request_id': 'req-be643d2d-1a33-40a1-8080-667ed6ab638d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:19.043825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] The token cache is being invalidate because project 02291b541fec4cf68e8e6c7564abec07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:19.044126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:19.044881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-be643d2d-1a33-40a1-8080-667ed6ab638d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:19.047402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 876/1727] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:18 2022] DELETE /identity/v3/projects/02291b541fec4cf68e8e6c7564abec07 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:19.052268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.052973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.053264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.109177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:19.115115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:19.283585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 852/1728] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:19.292059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.292255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.292609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.341283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:19.343211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:19.388660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 877/1729] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:19.392997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.433670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5d9a4-84c4-4f09-82aa-7697200a3429 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.451505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.452980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.453514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.454105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.455395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:19.456036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.460918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe91-3114-4424-8eac-72ce65bce58f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.504494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 853/1730] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:19.509938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.529250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bde9b7-397a-468b-9d0c-dd5cdc34b946 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.537658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.538363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.538826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.541247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.541987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.542278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.544311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.545055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.547801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.586898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 878/1731] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:19 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:19.589905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.607307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] MFA Rules not processed for user `aa56c83ab8494901b85d00d4f592e5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:19.609393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:19.686519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598bf2-5810-43ba-8df3-639fc164de1a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.695541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.696135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.696420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.696698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.697780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.700208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.737544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 879/1732] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:19 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:19.743081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.758128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1cbb4-b457-4fe3-8d39-8e74c20de86e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.758589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 854/1733] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:19.763491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.765196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.765786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.766082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.766360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.767394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.768964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:19.786805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.809838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 880/1734] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:19.813320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.832557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651b08-55f5-4280-a298-ccd4a392db02 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.844819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.845518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.845811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.846089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.846863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.848484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.875028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 881/1735] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:19.881614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:19.901159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8cb96-5820-421c-b245-6ff4af6f049a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.915623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.916527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.916810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.917088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] PATH_INFO: `/v3/projects/755eb970d2f1489188bc14440b15bb3c/users/08acb0a201424dcf96419b36854a278c/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.924699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cfb4f-ef31-47aa-a3fd-496c6bfb0623 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:19.939837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:19.940871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:19.941319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:19.941743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:19.960976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.962720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=755eb970d2f1489188bc14440b15bb3c, user_id=08acb0a201424dcf96419b36854a278c, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:19.964261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0a14a-204a-4f86-84cd-01e7181d5e2c service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.969119 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:19.979367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 855/1736] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:20.026168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 882/1737] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:19 2022] PUT /identity/v3/projects/755eb970d2f1489188bc14440b15bb3c/users/08acb0a201424dcf96419b36854a278c/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:20.031270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.032062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.032487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.067911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:20.072349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.078706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:20.081328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.257838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933d24-d602-4257-a477-0e26376c83aa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:20.259421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 856/1738] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:20.268670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:20.269885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.270267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.270814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] PATH_INFO: `/v3/projects/c16851676c6b4b81a8c64bf171e1768f` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.272167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16851676c6b4b81a8c64bf171e1768f)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:20.275729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:20.281843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:20.284198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.384238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16851676c6b4b81a8c64bf171e1768f', 'request_id': 'req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.392627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16851676c6b4b81a8c64bf171e1768f', 'request_id': 'req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.403360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16851676c6b4b81a8c64bf171e1768f', 'request_id': 'req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.420111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5883c9b-e4a6-4a5a-a3d5-e7c7c9e7d628 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:20.442564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] The token cache is being invalidate because project c16851676c6b4b81a8c64bf171e1768f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:20.443186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.444405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2768959e-36f7-4c1e-ad19-85fb2b3e0a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.451290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 883/1739] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:20 2022] DELETE /identity/v3/projects/c16851676c6b4b81a8c64bf171e1768f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:20.454967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.455507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.455959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.488473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:20.491628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.547599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:20.548418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.548710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.548988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.553623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.589174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 884/1740] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:20.589174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:20.589174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.600084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d835db9-c6c5-42fe-a177-1267c9b185b6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:20.605704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:20.606672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.606969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.607408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] PATH_INFO: `/v3/projects/72cd60cf3ecb4a448651d24b92fbe567` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.609660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72cd60cf3ecb4a448651d24b92fbe567)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:20.612026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:20.678574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72cd60cf3ecb4a448651d24b92fbe567', 'request_id': 'req-99d97d70-0e87-422a-ad5e-533da750cc12'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.684314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:20.686215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4390ec-23c8-4233-b817-ba486582978e service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:20.698277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72cd60cf3ecb4a448651d24b92fbe567', 'request_id': 'req-99d97d70-0e87-422a-ad5e-533da750cc12'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.700594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 857/1741] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:20.702799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.703391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.703914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.713466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72cd60cf3ecb4a448651d24b92fbe567', 'request_id': 'req-99d97d70-0e87-422a-ad5e-533da750cc12'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.732389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:20.735981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.741873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] The token cache is being invalidate because project 72cd60cf3ecb4a448651d24b92fbe567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:20.742177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.743995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-99d97d70-0e87-422a-ad5e-533da750cc12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:20.764493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 885/1742] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:20 2022] DELETE /identity/v3/projects/72cd60cf3ecb4a448651d24b92fbe567 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:20.766205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80dc2-6e37-444b-9fa2-9f76b97ccc7d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.766529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80dc2-6e37-444b-9fa2-9f76b97ccc7d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.766940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80dc2-6e37-444b-9fa2-9f76b97ccc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.812377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-20f80dc2-6e37-444b-9fa2-9f76b97ccc7d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:20.814035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-20f80dc2-6e37-444b-9fa2-9f76b97ccc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:20.871263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 858/1743] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:20.874643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:20.897959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125eb1aa-36b4-45a9-b562-f37f5e9e01b9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:20.908428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:20.910248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.910558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.910853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.912996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:20.913316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:20.919004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:20.950535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 859/1744] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:20.955467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:20.968698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc31e-34fb-4139-b787-b846b8f512da admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:20.973858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:20.974915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.975303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.975722 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:20.976581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:20.979336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:20.985197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 886/1745] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:20.995844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:20.996428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:20.996743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.022059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 860/1746] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:21.027941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.186504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709ece-bc90-4890-832a-8170784229db admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.195396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.196101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.196290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.196570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.197264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.199444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.220596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:21.221815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:21.242221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 887/1747] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:21.246315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.257348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96003c18-0e06-47ed-864c-bbfacbf9df6e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.259736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 861/1748] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:21.260993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.263493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.279285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.279285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.279285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.280278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:21.281659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.282789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01439f-8a29-447b-bf94-7cce7ffa82b5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.283617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.303948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 888/1749] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:21.307186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.307877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.308225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.308555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.309288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.310941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.311583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.330970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 862/1750] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:21.339475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.348782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6063-9b00-4c79-87a5-b871da7dce55 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.358505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c77e6e-b091-472d-9a6e-5f027ec32bb2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.363811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.364719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.364839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.365025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.366708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.367438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.370681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.373014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.373014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.373014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.377157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.379311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 863/1751] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:21.382144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.383326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.411862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b26d7-1e7a-4e58-958a-447e4f5c53a3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.419207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.420421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.420753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.421115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] PATH_INFO: `/v3/projects/487ae2cc07dd4c79bbf58716b43ddcfb/users/a3badf25f6874931af045f38088301a7/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.445028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=487ae2cc07dd4c79bbf58716b43ddcfb, user_id=a3badf25f6874931af045f38088301a7, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.448450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed680e04-bb86-4635-88b3-f805157cb1ab admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.489106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 864/1752] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:21 2022] PUT /identity/v3/projects/487ae2cc07dd4c79bbf58716b43ddcfb/users/a3badf25f6874931af045f38088301a7/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:21.489106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.489106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.489106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.490166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 889/1753] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/projects => generated 365 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:21.494429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.568177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] MFA Rules not processed for user `a3badf25f6874931af045f38088301a7`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:21.569990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:21.648079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6c03f-fc15-460b-a6c9-4c3cd044ba66 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.662967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.663671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.664534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.664534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.665557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.667257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.705951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 890/1754] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:21.710859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.740540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b42a2-f686-43ae-be2e-f164255e43f0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.748053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 865/1755] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:21.750806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.753936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.755033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.756427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.757017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.759802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.761980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.772779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf28b5-87a7-4464-95ef-ac7366bf7105 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.778617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.779587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.780077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.781152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.782204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.784386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:21.799203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 891/1756] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:21.801673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.848777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 866/1757] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:21.852877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:21.856407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f1e2-931d-42c3-a8e7-36477f082786 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.981855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc843-7f8a-4f92-9ef4-a53f484502af admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:21.983554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:21.983870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:21.984157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:21.984441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:21.985392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:21.986817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.001785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.002788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.003381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.003881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.006305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.010507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.011488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 892/1758] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:22.012375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.025699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2fa9-0017-4f16-b7d6-d890cf65c2b1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.033162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.035124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.035201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.035711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] PATH_INFO: `/v3/projects/f4e18384009d41adae4b37daa343d9b3/users/a729d07ff3af445e884cf3d4a1c3b16c/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.049734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 867/1759] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:21 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:22.053240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.066768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e18384009d41adae4b37daa343d9b3, user_id=a729d07ff3af445e884cf3d4a1c3b16c, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.069996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc678b2-de5a-456c-9417-65001a0e9eff admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.070669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.078165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.079108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.079456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.080477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.081507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.083854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefbf00-869a-460b-b144-236b6f3211d4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.093214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 893/1760] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:22 2022] PUT /identity/v3/projects/f4e18384009d41adae4b37daa343d9b3/users/a729d07ff3af445e884cf3d4a1c3b16c/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:22.097566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 868/1761] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:22.101564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.105977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.106415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.106954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.164240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] MFA Rules not processed for user `a729d07ff3af445e884cf3d4a1c3b16c`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:22.166738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:22.229048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b4f38-ed68-4363-8ef4-b41d584e6a25 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.237058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.237991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.238334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.238656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] PATH_INFO: `/v3/projects/496082bbcd714121bf637f976bc790cd/users/6ddd44b4898540f88b26f69dbaf7f52c/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.257637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496082bbcd714121bf637f976bc790cd, user_id=6ddd44b4898540f88b26f69dbaf7f52c, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.261500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.286446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 894/1762] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:22 2022] PUT /identity/v3/projects/496082bbcd714121bf637f976bc790cd/users/6ddd44b4898540f88b26f69dbaf7f52c/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:22.289281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.341534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 869/1763] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:22.346312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.365319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb35471-b074-4ed0-a609-4f9f22d0806c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.475937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.477570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.478694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.479188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.480320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.484010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.491939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57b53-23b6-4066-892c-8cf50f7df4b0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.510803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.511683 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.512256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.512579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.513437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.515162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.545676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 895/1764] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:22.555885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.575653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 870/1765] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:22 2022] POST /identity/v3/projects => generated 367 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:22.577010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.586419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8375-bca0-4441-9368-93926d76fdfb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.746371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.747304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.747590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.747867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] PATH_INFO: `/v3/projects/496082bbcd714121bf637f976bc790cd/users/6ddd44b4898540f88b26f69dbaf7f52c/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.749763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6796a5-1427-47d3-bce5-d6bdc16d3bd7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.754419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=496082bbcd714121bf637f976bc790cd, user_id=6ddd44b4898540f88b26f69dbaf7f52c, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.760145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.760852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.761137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.763458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af3fe1-f2b0-4bdc-9572-79d3af039fed admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.766272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.767057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.769023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.794442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 896/1766] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:22 2022] PUT /identity/v3/projects/496082bbcd714121bf637f976bc790cd/users/6ddd44b4898540f88b26f69dbaf7f52c/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:22.808272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.808561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.808840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.838795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 871/1767] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:22 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:22.842342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:22.857129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343a0c-8463-4e35-9d72-3620a6a356a8 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:22.866550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] MFA Rules not processed for user `6ddd44b4898540f88b26f69dbaf7f52c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:22.868522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:22.957787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:22.958762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:22.959133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:22.960028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:22.963732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:22.971895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:22.995808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 872/1768] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:22.997753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.015442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 897/1769] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:23.028247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.029887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:23.053546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e755-213b-4857-b7aa-2f7a0cbbabcf admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.072112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.073858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.076365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.076713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] PATH_INFO: `/v3/projects/52d984b7731e4c9d8f1960df91f4e209/users/dc50cab7f0684a0cbef913e3dc7cc6f8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.135881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d984b7731e4c9d8f1960df91f4e209, user_id=dc50cab7f0684a0cbef913e3dc7cc6f8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.151568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.163971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccf177-de7b-4581-ae05-a35f1adf3c45 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.174016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.174761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.176146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.176433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.191240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 873/1770] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:22 2022] PUT /identity/v3/projects/52d984b7731e4c9d8f1960df91f4e209/users/dc50cab7f0684a0cbef913e3dc7cc6f8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:23.198986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.303175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.304881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a06be-4f89-44dc-8c32-1e4505b01db4 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.321811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce193e7d-0a77-43f4-a574-769e56df8f4f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.334367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 898/1771] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:23.336124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.337354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.337800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.339475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.340344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.340737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.341065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.342800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.348562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.369324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 874/1772] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:23.371290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.391318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ffc9-f094-4f99-80b7-921c41b85080 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.403482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] MFA Rules not processed for user `08acb0a201424dcf96419b36854a278c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:23.404829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:23.415728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.417230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.417523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.417803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] PATH_INFO: `/v3/projects/52d984b7731e4c9d8f1960df91f4e209/users/dc50cab7f0684a0cbef913e3dc7cc6f8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.424717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d984b7731e4c9d8f1960df91f4e209, user_id=dc50cab7f0684a0cbef913e3dc7cc6f8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.436853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d082-bbf2-4557-9d1e-501b4c7ed272 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.474442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 875/1773] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:23 2022] PUT /identity/v3/projects/52d984b7731e4c9d8f1960df91f4e209/users/dc50cab7f0684a0cbef913e3dc7cc6f8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:23.485590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.486930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.487468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.567744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 899/1774] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:23.573576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.575971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:23.583118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] MFA Rules not processed for user `dc50cab7f0684a0cbef913e3dc7cc6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:23.585256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:23.607845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c4408-51d5-43ae-be2a-fa51509b55de None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.693088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 876/1775] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:23.705907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.707854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:23.771405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.772204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.774592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.774911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.775657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.787008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.848123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae1f3-7561-46ce-8096-051dc31fc0fa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:23.859266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:23.860611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.860897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.861231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.871380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 900/1776] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:23 2022] POST /identity/v3/projects => generated 371 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:23.875217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:23.876873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:23.878631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37043e6-e363-48fa-9b6f-859dc57c5316 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:23.905189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 877/1777] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:23.931118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:23.931442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:23.931869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:23.954120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] MFA Rules not processed for user `06701ca6a5e0427c9d5b540133c522a0`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:23.955399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:24.064433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7cb33-a561-450b-bfc3-cf7582ae93b7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.083492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.085238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.085614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.085952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.087374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.089480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.104093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 878/1778] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:24.104093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:24.122550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 901/1779] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:23 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:24.126335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:24.138272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aabb3-95c3-4bb2-874b-b52677b74f5d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.163204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.165152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.165449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.165782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.167707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.179561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.222946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 902/1780] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:24.226710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:24.244467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea0273-53e2-46a7-a883-d2c507f595d2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.252090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.252276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a723391-31f7-447c-aa37-a27260b80041 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.253989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.254478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.255035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.258754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.260038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.260038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.260223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] PATH_INFO: `/v3/projects/a6725ed7d7e5494db17c117859d5c07c/users/a9aaa2e4ebf043c8b4e6a90765060503/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.297994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6725ed7d7e5494db17c117859d5c07c, user_id=a9aaa2e4ebf043c8b4e6a90765060503, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.303661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec9f0e-f724-4510-9772-2e1eab852272 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.322893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 903/1781] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:24 2022] PUT /identity/v3/projects/a6725ed7d7e5494db17c117859d5c07c/users/a9aaa2e4ebf043c8b4e6a90765060503/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:24.325109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.325561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.325674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.332957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.334720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.344955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] MFA Rules not processed for user `aa56c83ab8494901b85d00d4f592e5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:24.345784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 879/1782] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:24.346252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:24.346753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:24.358983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed0ae-7c1c-4f4f-9c3f-d0f35f1459c3 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.435723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 904/1783] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:24.451027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.451952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.452363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.452780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.453637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.455393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518f44a-adbd-4af7-827b-334179d64f29 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.473983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:24.482152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 880/1784] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:24.494310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322212c-1ea3-42dd-94e7-3009da4237f0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:24.503611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:24.504521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:24.504802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:24.505074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] PATH_INFO: `/v3/projects/a6725ed7d7e5494db17c117859d5c07c/users/a9aaa2e4ebf043c8b4e6a90765060503/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:24.512494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6725ed7d7e5494db17c117859d5c07c, user_id=a9aaa2e4ebf043c8b4e6a90765060503, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:24.515182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:24.529576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 905/1785] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:24 2022] PUT /identity/v3/projects/a6725ed7d7e5494db17c117859d5c07c/users/a9aaa2e4ebf043c8b4e6a90765060503/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:25.793368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:25.793880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:25.795216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:25.828652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] MFA Rules not processed for user `a9aaa2e4ebf043c8b4e6a90765060503`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:25.830801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:25.915384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 881/1786] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:25.927304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:25.993329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4771-e38d-4b2a-947f-19406100161d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:25.997332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:25.998158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:25.998468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:25.998747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:26.083443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:26.084913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:26.113640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 906/1787] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:33.614966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:33.627504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a4868-6744-4189-ace4-4232f52bd674 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:33.631622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:33.632496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:33.632779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:33.633053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:33.728408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:33.730219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:33.743003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 882/1788] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:34.111663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:34.126976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f084f8-5542-46ce-adb9-8f8bc50a2fb8 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:34.133714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:34.135790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:34.137477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:34.137774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:34.162438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:34.163195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf3466-5dd6-4e0e-9e31-34cb117fe15c service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:34.174271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 907/1789] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:34.579569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:34.659105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096863-fc32-49a2-b56b-9d395047dca9 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:34.665073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:34.667257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:34.667698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:34.668105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] PATH_INFO: `/v3/users/a729d07ff3af445e884cf3d4a1c3b16c` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:34.669516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a729d07ff3af445e884cf3d4a1c3b16c)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:34.671736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:34.732697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a729d07ff3af445e884cf3d4a1c3b16c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:34.740201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a729d07ff3af445e884cf3d4a1c3b16c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:34.741373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a729d07ff3af445e884cf3d4a1c3b16c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:34.741734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a729d07ff3af445e884cf3d4a1c3b16c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:34.754340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a729d07ff3af445e884cf3d4a1c3b16c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:34.780081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 883/1790] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:34 2022] DELETE /identity/v3/users/a729d07ff3af445e884cf3d4a1c3b16c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:34.784419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:34.784745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:34.784972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:34.806418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:34.808598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:34.913065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 908/1791] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:34.942121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:34.943939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:35.043592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fdc5-c756-4558-b829-343c4ea88706 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:35.053329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:35.053708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.053988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.054423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.075882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:35.077285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:35.095767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 884/1792] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:35.469579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:35.471413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:35.487418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b96e4-cda5-4213-93e3-53683c8eb6da None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:35.493113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:35.494000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.494342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.494573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] PATH_INFO: `/v3/projects/f4e18384009d41adae4b37daa343d9b3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.496438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e18384009d41adae4b37daa343d9b3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:35.498261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:35.547445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e18384009d41adae4b37daa343d9b3', 'request_id': 'req-3bf75b71-b5cb-433e-b450-fea6662c6033'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.554668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e18384009d41adae4b37daa343d9b3', 'request_id': 'req-3bf75b71-b5cb-433e-b450-fea6662c6033'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.560797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e18384009d41adae4b37daa343d9b3', 'request_id': 'req-3bf75b71-b5cb-433e-b450-fea6662c6033'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.578588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] The token cache is being invalidate because project f4e18384009d41adae4b37daa343d9b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:35.578882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.579535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3bf75b71-b5cb-433e-b450-fea6662c6033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.581322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 909/1793] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:35 2022] DELETE /identity/v3/projects/f4e18384009d41adae4b37daa343d9b3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:35.585106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:35.596202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb16b9-d7f5-45e7-9134-9b3b00790d47 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:35.634348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddca6e-383f-499b-8780-a1323d2ab4b2 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.634742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddca6e-383f-499b-8780-a1323d2ab4b2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.635081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddca6e-383f-499b-8780-a1323d2ab4b2 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.636898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 910/1794] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 15:03:35.644494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.644824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.645270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.667558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:35.668818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:35.673292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:35.674337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.674704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.675173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] PATH_INFO: `/v3/users/dc50cab7f0684a0cbef913e3dc7cc6f8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.676632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc50cab7f0684a0cbef913e3dc7cc6f8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:35.678591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:35.731217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc50cab7f0684a0cbef913e3dc7cc6f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.738612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc50cab7f0684a0cbef913e3dc7cc6f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.739613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc50cab7f0684a0cbef913e3dc7cc6f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.740425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc50cab7f0684a0cbef913e3dc7cc6f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.743662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 911/1795] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:03:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:35.749961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc50cab7f0684a0cbef913e3dc7cc6f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:35.755856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:35.758231 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:35.772814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ec2ab-09e9-4b80-811f-a8b0bbb45d95 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:35.774971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 885/1796] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:35 2022] DELETE /identity/v3/users/dc50cab7f0684a0cbef913e3dc7cc6f8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:35.853874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:35.854567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:35.854855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:35.855134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:35.856910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:35.869158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:35.870796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543607b-e9fd-4b42-90e3-bb6601e2f3b9 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:35.881351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 912/1797] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:36.120755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.123061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:36.196419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794b42-c8ce-4ba7-b13b-626b3a8e5d58 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.203444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.204522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.205143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.205551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] PATH_INFO: `/v3/projects/52d984b7731e4c9d8f1960df91f4e209` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.207433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d984b7731e4c9d8f1960df91f4e209)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.209491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.268484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d984b7731e4c9d8f1960df91f4e209', 'request_id': 'req-620e8e83-bf96-4f03-ab36-93013ab74fb7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:36.277288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d984b7731e4c9d8f1960df91f4e209', 'request_id': 'req-620e8e83-bf96-4f03-ab36-93013ab74fb7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:36.284698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d984b7731e4c9d8f1960df91f4e209', 'request_id': 'req-620e8e83-bf96-4f03-ab36-93013ab74fb7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:36.300920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] The token cache is being invalidate because project 52d984b7731e4c9d8f1960df91f4e209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:36.301365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:36.302225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-620e8e83-bf96-4f03-ab36-93013ab74fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:36.303983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 886/1798] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:36 2022] DELETE /identity/v3/projects/52d984b7731e4c9d8f1960df91f4e209 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:36.318722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.318722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.318722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.333540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:36.334748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:36.394014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 913/1799] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:36.402981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.403378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.403664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.421816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:36.423012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:36.430757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 887/1800] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:36.434711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.443905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62235e-e9a1-4ea3-83ae-d5be5b67b95c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.448396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.449048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.449330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.449601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.450323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:36.450648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.452215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.463709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 914/1801] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:36.467644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.477420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53506c6-e416-486a-b9e5-59589c21bf53 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.481997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.482677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.482961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.483262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.483904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.485786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.510797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 888/1802] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:36 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:36.514884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.525222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538020b-792a-47b7-85f1-2d4e5b043c9a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.583385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.584062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.584344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.584618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.585301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.587781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.610156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 915/1803] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:36 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:36.611309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.622285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2159914-794c-43a1-9c23-eb9b7404e0c4 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.628174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.628979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.629266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.629527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.630255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.633249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.643958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 889/1804] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:36.647996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.658475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5652c68-9f29-4924-9982-7065810d5be2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.663030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.663686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.663975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.664252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.664967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.666580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d567a-b3ee-4346-b233-7a689742d511 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.676190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 916/1805] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:36.680280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.689894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb2c9-f116-4ec2-b5d9-54810017f3bb admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.694269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.695160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.695973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.696243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] PATH_INFO: `/v3/projects/5092a01452804314b3a7b6bb76ea8309/users/de9d4a95db0d47b58e3d29ff924f1492/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.709626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5092a01452804314b3a7b6bb76ea8309, user_id=de9d4a95db0d47b58e3d29ff924f1492, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.712927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.725143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 890/1806] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:36 2022] PUT /identity/v3/projects/5092a01452804314b3a7b6bb76ea8309/users/de9d4a95db0d47b58e3d29ff924f1492/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:36.736307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db30008-f5a1-4af0-a3ed-f1603c6d52b6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.736599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db30008-f5a1-4af0-a3ed-f1603c6d52b6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.736904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db30008-f5a1-4af0-a3ed-f1603c6d52b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.768301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0db30008-f5a1-4af0-a3ed-f1603c6d52b6 None None] MFA Rules not processed for user `de9d4a95db0d47b58e3d29ff924f1492`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:36.769424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0db30008-f5a1-4af0-a3ed-f1603c6d52b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:36.829472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 917/1807] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:36.840759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:36.842414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:36.894527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e53516-ac53-482c-9c28-99d9df512d38 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:36.898032 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:36.899297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:36.899576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:36.899846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:36.907708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:36.909125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899c27-206f-4496-844a-9be30b2b4c54 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:36.917854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 891/1808] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:37.480182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:37.480182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:37.480449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:37.482137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 918/1809] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:03:37.500194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:37.500194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:37.500194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:37.518124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:37.520134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:37.623672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 892/1810] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:03:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:37.740254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:37.743631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:37.765955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af216e3-2510-47e9-98b0-7ac64a615d73 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:37.773850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:37.774596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:37.775057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:37.775468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:37.785781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:37.788018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bef6b4-e05d-4059-bb59-fee6bd16d296 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:37.805249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 919/1811] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:47.398701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:47.469233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa9b5d-b34f-4fe4-8cf7-7bbacc18549d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:47.473711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:47.474613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:47.474896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:47.475170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] PATH_INFO: `/v3/users/de9d4a95db0d47b58e3d29ff924f1492` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:47.476363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9d4a95db0d47b58e3d29ff924f1492)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:47.478991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:47.522463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9d4a95db0d47b58e3d29ff924f1492'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:47.529445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9d4a95db0d47b58e3d29ff924f1492'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:47.530099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9d4a95db0d47b58e3d29ff924f1492'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:47.530701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9d4a95db0d47b58e3d29ff924f1492'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:47.538138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9d4a95db0d47b58e3d29ff924f1492'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:47.556561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 893/1812] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:47 2022] DELETE /identity/v3/users/de9d4a95db0d47b58e3d29ff924f1492 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:47.561132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:47.561675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:47.561675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:47.578775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:47.579997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:47.636780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 920/1813] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:47.646619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:47.648263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:47.703892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e967c21-1ae4-4361-be97-5024a2eaa4d4 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:47.709298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:47.709775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:47.710055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:47.710358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:47.724438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:47.724438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e76f7-5851-4cac-8bf1-6011c0b26cb0 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:47.733115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 894/1814] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:48.023556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.025620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:48.056226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ed153-577e-4f06-a403-784fd4e2c0e2 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.061125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.061983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.062403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.062733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] PATH_INFO: `/v3/projects/5092a01452804314b3a7b6bb76ea8309` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.063902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5092a01452804314b3a7b6bb76ea8309)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.065787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.112156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5092a01452804314b3a7b6bb76ea8309', 'request_id': 'req-75e1a5ea-47ca-4bad-8190-538f52bf1d75'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:48.118046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5092a01452804314b3a7b6bb76ea8309', 'request_id': 'req-75e1a5ea-47ca-4bad-8190-538f52bf1d75'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:48.124071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5092a01452804314b3a7b6bb76ea8309', 'request_id': 'req-75e1a5ea-47ca-4bad-8190-538f52bf1d75'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:48.135830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] The token cache is being invalidate because project 5092a01452804314b3a7b6bb76ea8309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:48.136114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:48.136789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-75e1a5ea-47ca-4bad-8190-538f52bf1d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:48.138437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 921/1815] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:48 2022] DELETE /identity/v3/projects/5092a01452804314b3a7b6bb76ea8309 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:48.143274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.143593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.143873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.161160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:48.162367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:48.220595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 895/1816] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:48.228172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.228455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.228727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.247012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:48.248198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:48.259740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 922/1817] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:48.265335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.274696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09801e-f0e7-46a9-99c7-07e27303c2b9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.279240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.279886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.280173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.280439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.281161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:48.281465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.283093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.294374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 896/1818] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:48.298733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.308183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde7f2f-da8b-45f6-b7e0-f7b5497947d7 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.312787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.313432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.313713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.313989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.314720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.316600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.345424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 923/1819] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:48.349694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.359436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017715-7931-4343-a870-007c6114f7c3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.414826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.416047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.416392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.416668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.417359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.419288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.442502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 897/1820] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:48.446767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.456596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbec275-1b1d-435a-912d-3f2266c13bfc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.461080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.461724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.462009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.462816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.463531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.465102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.477565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 924/1821] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:48.479182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.488748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accdb6-97a2-4dbc-90d1-db474017a69a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.493750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.494433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.494721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.494996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.495715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.497563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36b557-0707-4081-99d9-18873c12d5aa admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.511321 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 898/1822] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:48.512696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.522470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b988cd-5ae6-40da-85c0-31ac24d354a8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.529025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.530128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.530539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.530926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] PATH_INFO: `/v3/projects/5e16d8fd1fdc43f5ad07c20762089fd0/users/2699525301ad4328af2a0e7945864baf/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.547700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e16d8fd1fdc43f5ad07c20762089fd0, user_id=2699525301ad4328af2a0e7945864baf, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.551086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.566617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 925/1823] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:48 2022] PUT /identity/v3/projects/5e16d8fd1fdc43f5ad07c20762089fd0/users/2699525301ad4328af2a0e7945864baf/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:48.578693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.579104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.579504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.608108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] MFA Rules not processed for user `2699525301ad4328af2a0e7945864baf`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:48.609401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:48.699808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 899/1824] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:48.704617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.764669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bea3b-f671-478a-981c-ae9043bef69e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.769219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.769869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.770149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.770455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.771160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.773012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.799113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 926/1825] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:48.803175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.814411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fd376-23c5-45ce-9db0-93029706b64b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.868990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.869698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.871446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.871790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.872535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.874485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.893916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 900/1826] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:48 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:48.897689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.908356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dadef-1149-4219-8561-ea17b67c3df0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.912894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.913580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.913865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.914144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.914900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.917108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.927445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 927/1827] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:48.931478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.940958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566940-def4-4498-81ab-2a027c65e41b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:48.945418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:48.946289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:48.946636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:48.946907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] PATH_INFO: `/v3/projects/1102a4ef309145a899a482c4c18ee7b8/users/ce2e3cb319eb42a38d87f316b54bd370/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:48.960575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1102a4ef309145a899a482c4c18ee7b8, user_id=ce2e3cb319eb42a38d87f316b54bd370, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:48.963777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:48.975187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 901/1828] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:48 2022] PUT /identity/v3/projects/1102a4ef309145a899a482c4c18ee7b8/users/ce2e3cb319eb42a38d87f316b54bd370/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:48.979735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:48.989943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090ef22-4ddf-404e-b9ec-6a520545b9d7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:49.056553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:49.057270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:49.057554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:49.057829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:49.058613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:49.060220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3d4a-660d-4391-847f-01dfb7a6f6b8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:49.070932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 928/1829] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:49.075998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:49.086735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3446b-18d6-4245-9bc1-2faba7f611b2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:49.091562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:49.092460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:49.092738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:49.093010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] PATH_INFO: `/v3/projects/1102a4ef309145a899a482c4c18ee7b8/users/ce2e3cb319eb42a38d87f316b54bd370/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:49.095058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1102a4ef309145a899a482c4c18ee7b8, user_id=ce2e3cb319eb42a38d87f316b54bd370, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:49.097733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:49.108667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 902/1830] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:49 2022] PUT /identity/v3/projects/1102a4ef309145a899a482c4c18ee7b8/users/ce2e3cb319eb42a38d87f316b54bd370/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:49.119596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:49.119879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:49.120155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:49.147639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] MFA Rules not processed for user `ce2e3cb319eb42a38d87f316b54bd370`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:49.148841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:49.214979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 929/1831] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:49.225572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:49.227266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:49.283184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674169-c5bb-4577-bc46-59edfa6de38b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:49.286820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:49.287491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:49.287768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:49.288039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:49.342122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:49.343602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9704d1a-7028-40b7-8490-e30b509c5f81 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:49.353300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 903/1832] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:56.097062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:56.187100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9086-f444-4270-a92a-611c1878e7ca None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:56.191975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:56.193299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:56.193587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:56.193876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] PATH_INFO: `/v3/users/2699525301ad4328af2a0e7945864baf` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:56.195145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2699525301ad4328af2a0e7945864baf)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:56.197762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:56.242753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2699525301ad4328af2a0e7945864baf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:56.243458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2699525301ad4328af2a0e7945864baf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:56.250294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2699525301ad4328af2a0e7945864baf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:56.250952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2699525301ad4328af2a0e7945864baf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:56.258220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2699525301ad4328af2a0e7945864baf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:56.275681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 930/1833] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:56 2022] DELETE /identity/v3/users/2699525301ad4328af2a0e7945864baf => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:56.280314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:56.280854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:56.280997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:56.298305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:56.299519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:56.357591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 904/1834] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:56.367204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:56.369128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:56.424569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a39c9-d18c-44ac-90a4-cf3fb45697c6 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:56.428105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:56.428761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:56.429035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:56.429301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:56.438976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:56.440402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:56.450356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 931/1835] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:03:56.977228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:56.978904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:56.992152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30094eb0-18fc-404c-bcc0-e84faa9a7e08 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.007672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.008561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.009434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.009650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] PATH_INFO: `/v3/projects/5e16d8fd1fdc43f5ad07c20762089fd0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.011266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e16d8fd1fdc43f5ad07c20762089fd0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.013820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.065961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e16d8fd1fdc43f5ad07c20762089fd0', 'request_id': 'req-29299448-b47b-4896-9796-a5b036a6addc'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.072683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e16d8fd1fdc43f5ad07c20762089fd0', 'request_id': 'req-29299448-b47b-4896-9796-a5b036a6addc'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.079235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e16d8fd1fdc43f5ad07c20762089fd0', 'request_id': 'req-29299448-b47b-4896-9796-a5b036a6addc'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.094408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] The token cache is being invalidate because project 5e16d8fd1fdc43f5ad07c20762089fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:57.094769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.095480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.097362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 905/1836] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:56 2022] DELETE /identity/v3/projects/5e16d8fd1fdc43f5ad07c20762089fd0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:57.101082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.111150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d63194-95b4-467a-b04c-31d6566aaa3c service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.174591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.174591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.174591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.174591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] PATH_INFO: `/v3/users/ce2e3cb319eb42a38d87f316b54bd370` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.174924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2e3cb319eb42a38d87f316b54bd370)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.176645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.231098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2e3cb319eb42a38d87f316b54bd370'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.231898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2e3cb319eb42a38d87f316b54bd370'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.238467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2e3cb319eb42a38d87f316b54bd370'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.239109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2e3cb319eb42a38d87f316b54bd370'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.247517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b91cc004-d96c-4ad9-a531-f131343a3021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2e3cb319eb42a38d87f316b54bd370'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.268139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 932/1837] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:57 2022] DELETE /identity/v3/users/ce2e3cb319eb42a38d87f316b54bd370 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:57.281372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.282824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.295407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29299448-b47b-4896-9796-a5b036a6addc admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.353894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.354808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.355094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.355368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] PATH_INFO: `/v3/users/03c9259466f04438bad3ac99cc2af86b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.356592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c9259466f04438bad3ac99cc2af86b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.358560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.405132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c9259466f04438bad3ac99cc2af86b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.419153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c9259466f04438bad3ac99cc2af86b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.420239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c9259466f04438bad3ac99cc2af86b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.420873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c9259466f04438bad3ac99cc2af86b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.429401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c9259466f04438bad3ac99cc2af86b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.448016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 906/1838] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:57 2022] DELETE /identity/v3/users/03c9259466f04438bad3ac99cc2af86b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:57.452563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.452855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.453135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.470371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:57.471512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.531552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 933/1839] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:57.531740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.533331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.542289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.543849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.552210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb21e5-58ec-4872-9b77-df8553f63b82 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.558099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102391f7-6519-4376-aa56-385661689062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.559006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102391f7-6519-4376-aa56-385661689062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.559288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102391f7-6519-4376-aa56-385661689062 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.559560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102391f7-6519-4376-aa56-385661689062 admin admin] PATH_INFO: `/v3/projects/1102a4ef309145a899a482c4c18ee7b8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.560943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102391f7-6519-4376-aa56-385661689062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1102a4ef309145a899a482c4c18ee7b8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.562964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102391f7-6519-4376-aa56-385661689062 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.605511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42751ee6-0399-4e64-8b95-9ca71d25a971 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.611384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1102a4ef309145a899a482c4c18ee7b8', 'request_id': 'req-102391f7-6519-4376-aa56-385661689062'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.617192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1102a4ef309145a899a482c4c18ee7b8', 'request_id': 'req-102391f7-6519-4376-aa56-385661689062'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.625324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1102a4ef309145a899a482c4c18ee7b8', 'request_id': 'req-102391f7-6519-4376-aa56-385661689062'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.638204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] The token cache is being invalidate because project 1102a4ef309145a899a482c4c18ee7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:57.638553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.639201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-102391f7-6519-4376-aa56-385661689062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:57.640897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 907/1840] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:57 2022] DELETE /identity/v3/projects/1102a4ef309145a899a482c4c18ee7b8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:57.645938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b32b-d788-4af5-9967-b322679d65a3 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.646223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b32b-d788-4af5-9967-b322679d65a3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.646705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b32b-d788-4af5-9967-b322679d65a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.658826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.659479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.659765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.660039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.661818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.664396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e153b32b-d788-4af5-9967-b322679d65a3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:57.666064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e153b32b-d788-4af5-9967-b322679d65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.721458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 908/1841] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:57.729278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.729874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.729874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.730679 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.732125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.742894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 934/1842] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:57.748838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:57.750236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:57.758910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 909/1843] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:57.762912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.772099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1d9cc-898b-4cbb-9dab-f53755a8cd5b service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.777227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.777877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.778162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.778460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.779173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:57.779482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.781057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.793965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 935/1844] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:57.797713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.806815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0731094-25df-4bd4-8cdf-512263778073 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.812614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.813271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.813557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.814379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.814585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.816438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.842104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 910/1845] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:57 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:57.846638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.857708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920836e-f3dd-46a2-9505-0f91306c3903 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.928102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.929073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.929717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.930051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.931021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.934870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.966437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 936/1846] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:57 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:57.967679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:57.979379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b95695-a46d-43af-9c2a-273facd19e51 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:57.984227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:57.984889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:57.985207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:57.985463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:57.986179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:57.987795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:57.999215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 911/1847] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:58.003016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.014183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbd56e-0106-4f09-be2f-5289bbebe152 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.019268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.020013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.020360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.020700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.021645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.023449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1858f-e4e1-4ee7-a172-241c0f47b657 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.037751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 937/1848] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:58.038928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.049468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14034cea-53c9-43b5-a4ba-4d5000985866 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.054713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.055608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.056137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.056417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] PATH_INFO: `/v3/projects/a98cae0b68ed49fcb21905b39b8d4287/users/e4147016621e4c20b5e66a41440ed69b/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.072164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98cae0b68ed49fcb21905b39b8d4287, user_id=e4147016621e4c20b5e66a41440ed69b, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.074867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.094027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 912/1849] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:58 2022] PUT /identity/v3/projects/a98cae0b68ed49fcb21905b39b8d4287/users/e4147016621e4c20b5e66a41440ed69b/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:58.101604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.102148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.102284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.125437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.127863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.133562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] MFA Rules not processed for user `e4147016621e4c20b5e66a41440ed69b`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:58.135116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.195421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a9cec-ad2d-476e-aa07-a1b710552b1c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.200317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.201249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.201594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.201929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] PATH_INFO: `/v3/projects/0612cad27c8e4f77ad14bd9d10eba2b6` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.203209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0612cad27c8e4f77ad14bd9d10eba2b6)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.205184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.219347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 938/1850] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:58.220448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.233204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ba21d-7b95-4a3a-8628-9284a1aed736 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.239650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.240377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.240665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.240988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.241791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.243863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.255723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0612cad27c8e4f77ad14bd9d10eba2b6', 'request_id': 'req-98c0e104-88b0-43eb-8eb0-f4d3072e810c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.262800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0612cad27c8e4f77ad14bd9d10eba2b6', 'request_id': 'req-98c0e104-88b0-43eb-8eb0-f4d3072e810c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.270805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0612cad27c8e4f77ad14bd9d10eba2b6', 'request_id': 'req-98c0e104-88b0-43eb-8eb0-f4d3072e810c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.288871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 939/1851] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:58.289031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] The token cache is being invalidate because project 0612cad27c8e4f77ad14bd9d10eba2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:58.289489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.290708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.292318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 913/1852] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:58 2022] DELETE /identity/v3/projects/0612cad27c8e4f77ad14bd9d10eba2b6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:58.294461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.296292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.296530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.298070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.370850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf82674-2f3a-4d98-b7d6-55e49847477c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.372454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e104-88b0-43eb-8eb0-f4d3072e810c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.375799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.376464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.376741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.377009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.377161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.377690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.378031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.378316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.378621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] PATH_INFO: `/v3/users/f41a0749a38f45ebacc134cfb64dfb19` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.379755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.379868 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41a0749a38f45ebacc134cfb64dfb19)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.381784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.403431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 940/1853] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:58.404664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.414752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb64641-4000-4b42-a976-4a28394ad97a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.419466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.420142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.420429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.420705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.421418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.423064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.428129 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41a0749a38f45ebacc134cfb64dfb19'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.432988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 941/1854] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:58.434774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41a0749a38f45ebacc134cfb64dfb19'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.435540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41a0749a38f45ebacc134cfb64dfb19'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.436166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41a0749a38f45ebacc134cfb64dfb19'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.436804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.438445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.443766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41a0749a38f45ebacc134cfb64dfb19'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.451676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582843b0-fdb1-4c99-889c-bfe9e7791e32 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.456555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.457223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.457506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.457781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.458519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.460137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921e13c-a733-477a-8073-580d757e8337 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.461266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 914/1855] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:03:58 2022] DELETE /identity/v3/users/f41a0749a38f45ebacc134cfb64dfb19 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:58.471373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 942/1856] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:58.475508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.538736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4e39-fb75-407b-ac3d-0977aed365c1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.544761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.545661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.545978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.546383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] PATH_INFO: `/v3/projects/b1e5a893a5f54bcab99f20f9bb3087e7/users/9ca02256343b4f04860dacc9791be3ef/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.563280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e5a893a5f54bcab99f20f9bb3087e7, user_id=9ca02256343b4f04860dacc9791be3ef, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.565943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.579125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 915/1857] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:58 2022] PUT /identity/v3/projects/b1e5a893a5f54bcab99f20f9bb3087e7/users/9ca02256343b4f04860dacc9791be3ef/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:58.589717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.590083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.590479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.622261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] MFA Rules not processed for user `9ca02256343b4f04860dacc9791be3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:58.623539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.693757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 943/1858] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:58.699034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.730012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.731465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.766761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb388e2-10cd-4fc9-9697-6fb94543505a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.771620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.772279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.772564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.772839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.773525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.775527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.799513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b50fd-770e-4a2d-b8d5-cdf172e8da11 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.799851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 916/1859] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:58.803860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.852442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.853338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.853613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.853886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] PATH_INFO: `/v3/projects/5a30ce85d2504da9af7adada79464d3b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.855072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a30ce85d2504da9af7adada79464d3b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.856959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.861709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd0984-522b-430b-aa12-9a8cce316658 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.866216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.866968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.867248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.867520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.868705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.870626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.890897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 917/1860] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/users => generated 298 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:03:58.894688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.902989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a30ce85d2504da9af7adada79464d3b', 'request_id': 'req-a3f54748-6ed9-4b92-a02e-beac2c110a39'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.905035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf949ba-8f2b-45d2-be41-8d423cf39b63 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:58.909630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a30ce85d2504da9af7adada79464d3b', 'request_id': 'req-a3f54748-6ed9-4b92-a02e-beac2c110a39'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.915120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a30ce85d2504da9af7adada79464d3b', 'request_id': 'req-a3f54748-6ed9-4b92-a02e-beac2c110a39'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.927493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] The token cache is being invalidate because project 5a30ce85d2504da9af7adada79464d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:03:58.927775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.928410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3f54748-6ed9-4b92-a02e-beac2c110a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:03:58.929965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 944/1861] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:03:58 2022] DELETE /identity/v3/projects/5a30ce85d2504da9af7adada79464d3b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:58.935394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762654-02e0-4943-a534-42784624e4f5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.935745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762654-02e0-4943-a534-42784624e4f5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.936024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762654-02e0-4943-a534-42784624e4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.961628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2b762654-02e0-4943-a534-42784624e4f5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:58.963068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2b762654-02e0-4943-a534-42784624e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:58.975607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:58.976281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:58.976561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:58.976970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:58.978309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:58.980075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:58.991907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 918/1862] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:58.997036 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:58.999060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.012006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f870-0595-4f96-bd4a-5b3af0687736 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.016917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.017830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.018109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.018499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] PATH_INFO: `/v3/projects/a479675e5da04e71aa05f96b99719fe8/users/0f95da40f631454893dcf070b797ea1d/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.031193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 945/1863] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:59.034004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a479675e5da04e71aa05f96b99719fe8, user_id=0f95da40f631454893dcf070b797ea1d, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.038369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.038611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.038900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.039226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.055614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 919/1864] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:58 2022] PUT /identity/v3/projects/a479675e5da04e71aa05f96b99719fe8/users/0f95da40f631454893dcf070b797ea1d/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:59.059213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:59.059847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.060485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.123627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 946/1865] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:59.128022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.131275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b5257-cebc-4487-9afa-d75a60d45b67 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.136891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.137049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd6e8a-13ba-421b-9867-ddfb6de67001 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.137156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.137156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.138759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.139763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.140632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.141280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.141563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.141838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.141961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.142590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:03:59.142890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.144475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.155000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 920/1866] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:59.158925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.162569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 947/1867] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:03:59.163619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.169495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9bfdf-778d-4f6c-b621-6a05af278251 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.174280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.175284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.175659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.175952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93168040-ce10-4541-bde8-7700b5637250 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.176125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] PATH_INFO: `/v3/projects/a479675e5da04e71aa05f96b99719fe8/users/0f95da40f631454893dcf070b797ea1d/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.178185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a479675e5da04e71aa05f96b99719fe8, user_id=0f95da40f631454893dcf070b797ea1d, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.181403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.181569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbc9b1-0209-4ac7-8f5b-e0b330398df9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.182111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.182415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.182706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.183399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.185291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.192639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 921/1868] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:59 2022] PUT /identity/v3/projects/a479675e5da04e71aa05f96b99719fe8/users/0f95da40f631454893dcf070b797ea1d/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:59.203075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.203198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.203575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.226665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 948/1869] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:59.231216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.236933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] MFA Rules not processed for user `0f95da40f631454893dcf070b797ea1d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:59.238144 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.304579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e73c9-a16e-4b53-82bf-9159ae62d224 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.313140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.314209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.314538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.314802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.315612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.317503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.324252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 922/1870] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:03:59.335733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.337236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.341245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 949/1871] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:59.342338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.351851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4fab8-d61b-4f40-bece-bfad560a5419 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.357470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.358161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.358477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.358747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.359478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.361038 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.372287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 950/1872] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:59.375923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.385682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18ca20-14f2-4c14-b112-e34c61c095ba admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.390419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.391082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.391364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.391638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.392379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.393967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.396110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193df3f4-b075-4846-b348-727521eb74a2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.400443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.401223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.401596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.401988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.403874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.405047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 951/1873] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:59.409033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.419302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6200-7586-490d-928b-748d00a0d596 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.424105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.424983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.425264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.425537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] PATH_INFO: `/v3/projects/41edeacb10e44c678f18a00af8fbac2b/users/01f12031bb2147e09d1f35025171f466/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.440050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41edeacb10e44c678f18a00af8fbac2b, user_id=01f12031bb2147e09d1f35025171f466, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.442986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c72ba-0f26-4238-b482-c5b6ce56585b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.456698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 952/1874] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:59 2022] PUT /identity/v3/projects/41edeacb10e44c678f18a00af8fbac2b/users/01f12031bb2147e09d1f35025171f466/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:03:59.464933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.466998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.471823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.471823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.472022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.479735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 923/1875] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:03:59.500608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] MFA Rules not processed for user `01f12031bb2147e09d1f35025171f466`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:03:59.502233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:03:59.574476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 953/1876] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:03:59.575784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.586348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff5a6b-3ba9-43da-b467-428e89781842 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.648932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.650162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.650477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.650756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.651449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.653350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.679834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 924/1877] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:59.684035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.695080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe972a-2323-42a3-92ae-5cafa2410526 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.759994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.760927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.761277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.761675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.762629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.764662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.796988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 954/1878] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:03:59 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:03:59.798169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.808461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332849-b378-48f8-bdf6-80c523af8eda admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.813040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.813699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.813982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.814256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.815094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.816586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.827497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 925/1879] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:03:59.831580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.841723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f024c3-ad2f-46db-8a3f-ae137052208d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.846315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.847209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.847483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.847756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] PATH_INFO: `/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9/users/edf658581d6c4e02a3b28b093cacb5ea/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.861608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e929e0ef598e4d868a1e3df9ce134ee9, user_id=edf658581d6c4e02a3b28b093cacb5ea, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.864264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.876282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 955/1880] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:59 2022] PUT /identity/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9/users/edf658581d6c4e02a3b28b093cacb5ea/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:03:59.880040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.890222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f8ab-bbaa-45d0-b0aa-5d44628bcd32 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.943563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.944571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.944571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.944778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.945493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.947659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e50a4-c952-465b-9939-4d410aca4598 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.960873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 926/1881] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:03:59.965891 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:03:59.975808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df00a4-1886-4e7c-a6f5-4179c19c17ac admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:03:59.980290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:03:59.981161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:03:59.981441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:03:59.981712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] PATH_INFO: `/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9/users/edf658581d6c4e02a3b28b093cacb5ea/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:03:59.983802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e929e0ef598e4d868a1e3df9ce134ee9, user_id=edf658581d6c4e02a3b28b093cacb5ea, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:03:59.986660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:03:59.997047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 956/1882] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:03:59 2022] PUT /identity/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9/users/edf658581d6c4e02a3b28b093cacb5ea/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:00.007111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:00.007403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:00.007684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:00.036734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] MFA Rules not processed for user `edf658581d6c4e02a3b28b093cacb5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:00.038061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:00.104270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 927/1883] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:00.115365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:00.117066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:00.173395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134154c-0c3e-42fb-8766-fccedac07af1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:00.177467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:00.178338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:00.178814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:00.179196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:00.189538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:00.191054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:00.201314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 957/1884] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:00.317918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:00.330271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336dd9-6ff5-498b-b60f-26ecfdceb7b3 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:00.336534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:00.339572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:00.339963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:00.340123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:00.442831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:00.444467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:00.457366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 928/1885] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:00.980557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:00.988614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a5bf-55ca-4a0a-8758-56cab7876fc7 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:00.992405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:00.993041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:00.993354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:00.993624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:00.995412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:01.052956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:01.054372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:01.063334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 958/1886] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:01.740183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:01.755494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef108c-f66c-4f48-9a48-c03ed9123530 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:01.760361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:01.761007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:01.761281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:01.761703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:01.844830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:01.846284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:01.859546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 929/1887] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:08.741881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:08.743815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:08.834930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6142bf4-b2fe-4a25-a4fc-ae73b873afa2 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:08.847140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:08.848043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:08.848327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:08.848610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] PATH_INFO: `/v3/users/a3badf25f6874931af045f38088301a7` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:08.849841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3badf25f6874931af045f38088301a7)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:08.851882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:08.953114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3badf25f6874931af045f38088301a7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:08.953988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3badf25f6874931af045f38088301a7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:08.969405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3badf25f6874931af045f38088301a7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:08.970687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:08.976365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3badf25f6874931af045f38088301a7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:08.985847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0d2-c8a2-48bf-b819-58d236fd6b3a service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:08.988007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e268f850-9a95-4a28-88f9-c0789f537909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3badf25f6874931af045f38088301a7'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:08.992434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:08.993426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:08.993997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:08.994491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] PATH_INFO: `/v3/users/e4147016621e4c20b5e66a41440ed69b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:08.995835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4147016621e4c20b5e66a41440ed69b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:08.999675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:09.012467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 959/1888] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:08 2022] DELETE /identity/v3/users/a3badf25f6874931af045f38088301a7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:09.017480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.018028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.018425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.047114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:09.049358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:09.095120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4147016621e4c20b5e66a41440ed69b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.111101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4147016621e4c20b5e66a41440ed69b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.112313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4147016621e4c20b5e66a41440ed69b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.113144 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4147016621e4c20b5e66a41440ed69b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.123389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d020ca18-b327-4049-b9b8-fb0166ec05d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4147016621e4c20b5e66a41440ed69b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.150279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 930/1889] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:08 2022] DELETE /identity/v3/users/e4147016621e4c20b5e66a41440ed69b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:09.171499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.172749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.173224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.174976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 960/1890] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:09.193791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:09.195333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:09.205943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:09.207653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:09.282047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7711-5bcd-4dc7-84cd-e20ff88aaced None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:09.295485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 931/1891] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:09.307746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:09.309208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:09.349163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:09.350561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.350561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.351212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.362477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:09.364834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:09.388711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 961/1892] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:09.393441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64231e0-22b7-417c-bf13-833a3f45f40a None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:09.397781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:09.398653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.399923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.399923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.408879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:09.410318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:09.422310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 932/1893] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:09.807246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:09.811563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:09.831176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb522-f302-4c93-9d9f-7fead160e5c1 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:09.836284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:09.837607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.837954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.838293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] PATH_INFO: `/v3/projects/487ae2cc07dd4c79bbf58716b43ddcfb` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.839747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=487ae2cc07dd4c79bbf58716b43ddcfb)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:09.841905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:09.890490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487ae2cc07dd4c79bbf58716b43ddcfb', 'request_id': 'req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.896830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487ae2cc07dd4c79bbf58716b43ddcfb', 'request_id': 'req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.903103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487ae2cc07dd4c79bbf58716b43ddcfb', 'request_id': 'req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.916946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] The token cache is being invalidate because project 487ae2cc07dd4c79bbf58716b43ddcfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:09.917249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.917920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:09.919682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 962/1894] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:09 2022] DELETE /identity/v3/projects/487ae2cc07dd4c79bbf58716b43ddcfb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:09.924521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:09.935922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c549e-d52a-48df-a233-f51a85096bc5 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:09.993394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:09.994181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:09.994497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:09.994779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] PATH_INFO: `/v3/users/6ddd44b4898540f88b26f69dbaf7f52c` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:09.995968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddd44b4898540f88b26f69dbaf7f52c)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:09.998325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:10.047010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddd44b4898540f88b26f69dbaf7f52c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.053866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddd44b4898540f88b26f69dbaf7f52c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.054571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddd44b4898540f88b26f69dbaf7f52c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.055185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddd44b4898540f88b26f69dbaf7f52c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.062487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5f89d1a9-e034-4079-a7c9-5c6fdbb3fd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddd44b4898540f88b26f69dbaf7f52c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.081359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 933/1895] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:09 2022] DELETE /identity/v3/users/6ddd44b4898540f88b26f69dbaf7f52c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:10.364690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:10.366938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:10.386664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ccc07-e8bd-44e3-aa2f-b2f50672bfbb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:10.470551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:10.471332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.471875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.472158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] PATH_INFO: `/v3/projects/496082bbcd714121bf637f976bc790cd` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.474268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] RBAC: Authorizing `identity:delete_project(project_id=496082bbcd714121bf637f976bc790cd)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:10.475877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:10.530849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496082bbcd714121bf637f976bc790cd', 'request_id': 'req-8f57b8ce-85c1-4372-8699-77dffa3d727a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.537971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496082bbcd714121bf637f976bc790cd', 'request_id': 'req-8f57b8ce-85c1-4372-8699-77dffa3d727a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.557417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496082bbcd714121bf637f976bc790cd', 'request_id': 'req-8f57b8ce-85c1-4372-8699-77dffa3d727a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.577281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] The token cache is being invalidate because project 496082bbcd714121bf637f976bc790cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:10.577620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.578584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f57b8ce-85c1-4372-8699-77dffa3d727a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:10.580505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 963/1896] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:10 2022] DELETE /identity/v3/projects/496082bbcd714121bf637f976bc790cd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:10.586150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.588466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.589952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.610524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:10.611824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:10.681167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 934/1897] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:10.690154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.690658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.690937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.719544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:10.721348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:10.735685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 964/1898] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:10.740264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:10.750347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a85fd5-a54e-4b8e-acbe-f1ed7dd4ffda None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:10.756058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:10.756655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.756936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.757223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.760315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:10.760641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:10.762273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:10.784536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 935/1899] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:10.786065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:10.796159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88eba2-7e7c-464f-b5d7-75e482e85b03 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:10.803181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:10.804088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.804502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.804770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.806804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:10.808406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:10.844239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 965/1900] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:10 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:10.848499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:10.859694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2468f-252b-4365-a67f-8507a3072bce admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:10.939112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:10.939901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.940299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.940672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.941462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:10.943938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:10.971940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 936/1901] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:10 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:10.975771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:10.989488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79709d-cbcb-4e27-861e-f57f90917498 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:10.995595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:10.996600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:10.996923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:10.997249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:10.998146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:10.999886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.009594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.013413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.024516 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 966/1902] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:11.025616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.028138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9f2c4-fdc7-4b0d-b59d-5c4734672d12 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.034095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.035612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.036130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.036715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] PATH_INFO: `/v3/projects/a98cae0b68ed49fcb21905b39b8d4287` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.038338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98cae0b68ed49fcb21905b39b8d4287)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.040842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29567d4d-8569-4d7d-9e35-50e8d2aa1fd2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.041497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.046850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.047919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.048270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.048609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.049412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.065955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.094090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 967/1903] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:11.094753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.120517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98cae0b68ed49fcb21905b39b8d4287', 'request_id': 'req-ccece6a5-95f1-48d1-9727-5471a29c9aa9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.122717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54703019-b76e-47d2-8eb3-f509b855e29e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.127492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98cae0b68ed49fcb21905b39b8d4287', 'request_id': 'req-ccece6a5-95f1-48d1-9727-5471a29c9aa9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.139327 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98cae0b68ed49fcb21905b39b8d4287', 'request_id': 'req-ccece6a5-95f1-48d1-9727-5471a29c9aa9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.159427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] The token cache is being invalidate because project a98cae0b68ed49fcb21905b39b8d4287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:11.159713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.160777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.163443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 937/1904] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:11 2022] DELETE /identity/v3/projects/a98cae0b68ed49fcb21905b39b8d4287 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:11.167822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.169226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.224264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.229163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.229616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.230062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] PATH_INFO: `/v3/projects/0b7e984f320143279dcbad4d170c8c8b/users/c738eb2922d14b4b915ae1820c20fa42/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.261252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7e984f320143279dcbad4d170c8c8b, user_id=c738eb2922d14b4b915ae1820c20fa42, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.266181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23d1a9-17e2-4bc8-9742-128a8adbd5a4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.273484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece6a5-95f1-48d1-9727-5471a29c9aa9 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.280317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.281402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.282251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.282718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] PATH_INFO: `/v3/users/9ca02256343b4f04860dacc9791be3ef` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.284116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 968/1905] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:11 2022] PUT /identity/v3/projects/0b7e984f320143279dcbad4d170c8c8b/users/c738eb2922d14b4b915ae1820c20fa42/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:11.284566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca02256343b4f04860dacc9791be3ef)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.287009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.296244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.296427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.296672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.326968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] MFA Rules not processed for user `c738eb2922d14b4b915ae1820c20fa42`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:11.328314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.385948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca02256343b4f04860dacc9791be3ef'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.403889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca02256343b4f04860dacc9791be3ef'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.404623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca02256343b4f04860dacc9791be3ef'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.405240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca02256343b4f04860dacc9791be3ef'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.415372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca02256343b4f04860dacc9791be3ef'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.415826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 969/1906] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:11.416860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.418295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.433032 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 938/1907] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:11 2022] DELETE /identity/v3/users/9ca02256343b4f04860dacc9791be3ef => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:11.434610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.436676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.507771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8c176-5d3b-470b-9af7-585080e3e924 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.539394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab1ceb-76ed-44d3-85ce-2c7709058bf2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.546658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.547521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.548279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.550891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.550891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.568960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.575805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.576765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.577141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.577488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] PATH_INFO: `/v3/users/08acb0a201424dcf96419b36854a278c` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.579262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08acb0a201424dcf96419b36854a278c)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.581551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.615782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 939/1908] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:11 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:11.619806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.651508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08acb0a201424dcf96419b36854a278c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.652205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08acb0a201424dcf96419b36854a278c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.665767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08acb0a201424dcf96419b36854a278c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.667602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08acb0a201424dcf96419b36854a278c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.680993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08acb0a201424dcf96419b36854a278c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.702294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 970/1909] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:11 2022] DELETE /identity/v3/users/08acb0a201424dcf96419b36854a278c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:11.719180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fc73-28c0-4281-a5ef-3c1f33ec44af admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.784390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.786554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.790502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.791466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.791862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.792433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.793250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.795621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.802330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae9c74-01d4-4201-a527-f39e5a411196 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.808201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.809568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.810171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.810476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] PATH_INFO: `/v3/users/01f12031bb2147e09d1f35025171f466` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.811961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f12031bb2147e09d1f35025171f466)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.815535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.823230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 940/1910] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:11 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:11.828037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.830734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.845603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54ee-dceb-48ad-b978-b24bcb634879 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.851718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.853540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.853977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.854553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.855793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.858436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.871727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 941/1911] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:11.875711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.895215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7e6f-db6c-47f4-ad9d-531437dfd6c5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:11.897143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f12031bb2147e09d1f35025171f466'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.897962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f12031bb2147e09d1f35025171f466'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.907681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:11.908667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:11.908938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:11.910710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] PATH_INFO: `/v3/projects/bad828b2dd944740ac938dbf6373d8ea/users/88442e67421b41dd9ea10e036b09e1c4/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:11.920772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f12031bb2147e09d1f35025171f466'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.921528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f12031bb2147e09d1f35025171f466'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.932518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f12031bb2147e09d1f35025171f466'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:11.946718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad828b2dd944740ac938dbf6373d8ea, user_id=88442e67421b41dd9ea10e036b09e1c4, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:11.949857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31986a2-e839-4fd4-9e80-0247fe005f2a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:11.961373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 971/1912] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:11 2022] DELETE /identity/v3/users/01f12031bb2147e09d1f35025171f466 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:11.963229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:11.967601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:11.995504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 942/1913] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:11 2022] PUT /identity/v3/projects/bad828b2dd944740ac938dbf6373d8ea/users/88442e67421b41dd9ea10e036b09e1c4/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:11.999023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.000119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.001611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.028068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:12.030651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.108371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6596e5-56fd-411c-ade0-399478e9965a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.143895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 943/1914] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:12.145018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.146447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.169447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707192c-2995-4b30-b05b-8a0e2d73f12b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.185170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.185789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.186141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.186786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.187340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.189654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.209625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 944/1915] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:12.210810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.214355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.215860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.216742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.217055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.217325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] PATH_INFO: `/v3/projects/755eb970d2f1489188bc14440b15bb3c` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.218552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755eb970d2f1489188bc14440b15bb3c)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.220487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.290758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2208a9-18df-4456-ab59-8a2b4e41bb56 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.302425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755eb970d2f1489188bc14440b15bb3c', 'request_id': 'req-2aa2f20e-684e-4dea-a2a1-d157f691da44'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.330942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755eb970d2f1489188bc14440b15bb3c', 'request_id': 'req-2aa2f20e-684e-4dea-a2a1-d157f691da44'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.357300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755eb970d2f1489188bc14440b15bb3c', 'request_id': 'req-2aa2f20e-684e-4dea-a2a1-d157f691da44'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.384534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] The token cache is being invalidate because project 755eb970d2f1489188bc14440b15bb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:12.384534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.384534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.390441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 972/1916] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:11 2022] DELETE /identity/v3/projects/755eb970d2f1489188bc14440b15bb3c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:12.390441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.391339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.393130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.393874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.394163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.394466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.396276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.508715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2f20e-684e-4dea-a2a1-d157f691da44 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.514788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.515981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.516344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.516694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] PATH_INFO: `/v3/projects/bad828b2dd944740ac938dbf6373d8ea/users/88442e67421b41dd9ea10e036b09e1c4/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.518962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad828b2dd944740ac938dbf6373d8ea, user_id=88442e67421b41dd9ea10e036b09e1c4, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.521756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.526616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.528070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.542518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 973/1917] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:12 2022] PUT /identity/v3/projects/bad828b2dd944740ac938dbf6373d8ea/users/88442e67421b41dd9ea10e036b09e1c4/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:12.542712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 945/1918] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:12.542795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.542869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.543277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.545360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.558707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a953-f494-44f6-9fe4-9910c9d8c62b service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.633723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.635086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.635370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.635777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbb9f-6364-47c2-8c65-c02d7b9eb3ed admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.636684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] PATH_INFO: `/v3/users/a9aaa2e4ebf043c8b4e6a90765060503` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.640074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9aaa2e4ebf043c8b4e6a90765060503)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.643577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.648890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:12.651173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.651453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.651748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] PATH_INFO: `/v3/projects/b1e5a893a5f54bcab99f20f9bb3087e7` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.653701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e5a893a5f54bcab99f20f9bb3087e7)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:12.656736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:12.733905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9aaa2e4ebf043c8b4e6a90765060503'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.735898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e5a893a5f54bcab99f20f9bb3087e7', 'request_id': 'req-8420eea1-3749-475d-90f3-80c2a5cd46fc'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.741715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9aaa2e4ebf043c8b4e6a90765060503'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.742806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9aaa2e4ebf043c8b4e6a90765060503'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.744128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9aaa2e4ebf043c8b4e6a90765060503'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.745627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e5a893a5f54bcab99f20f9bb3087e7', 'request_id': 'req-8420eea1-3749-475d-90f3-80c2a5cd46fc'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.753842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-025456a1-b440-4a7c-8f3a-8e0beffac173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9aaa2e4ebf043c8b4e6a90765060503'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.754105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e5a893a5f54bcab99f20f9bb3087e7', 'request_id': 'req-8420eea1-3749-475d-90f3-80c2a5cd46fc'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.775626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 946/1919] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:12 2022] DELETE /identity/v3/users/a9aaa2e4ebf043c8b4e6a90765060503 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:12.777197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:12.777701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:12.777977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:12.781790 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] The token cache is being invalidate because project b1e5a893a5f54bcab99f20f9bb3087e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:12.782333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.783879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:12.786890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 974/1920] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:12 2022] DELETE /identity/v3/projects/b1e5a893a5f54bcab99f20f9bb3087e7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:12.792017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.794088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.807575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] MFA Rules not processed for user `88442e67421b41dd9ea10e036b09e1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:12.808844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:12.897063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420eea1-3749-475d-90f3-80c2a5cd46fc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:12.898533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 947/1921] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:12.912673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:12.916291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:13.013318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.016580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.016873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.017159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] PATH_INFO: `/v3/users/0f95da40f631454893dcf070b797ea1d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.020915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f95da40f631454893dcf070b797ea1d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.025453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.028742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb90bbd-3dd6-41b3-99af-cf1dc7416578 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:13.040453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.040453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.040453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.040453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.061848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.063869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.091212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f95da40f631454893dcf070b797ea1d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.091447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 948/1922] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:13.093154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f95da40f631454893dcf070b797ea1d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.103251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f95da40f631454893dcf070b797ea1d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.104184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f95da40f631454893dcf070b797ea1d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.115161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f95da40f631454893dcf070b797ea1d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.172747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 975/1923] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:12 2022] DELETE /identity/v3/users/0f95da40f631454893dcf070b797ea1d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:13.221898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:13.231666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339eea-7e6f-49d9-a905-f4bff99ce11f service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:13.293037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.293705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.293864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.294141 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.296239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:13.303463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:13.305364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:13.367357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.369444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.381300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 949/1924] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:13.439237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370d643-7d06-46dc-a9d7-6c06301e314d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:13.445705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.447902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.448264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.448617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] PATH_INFO: `/v3/projects/41edeacb10e44c678f18a00af8fbac2b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.449968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41edeacb10e44c678f18a00af8fbac2b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.453892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.521103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41edeacb10e44c678f18a00af8fbac2b', 'request_id': 'req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.543648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41edeacb10e44c678f18a00af8fbac2b', 'request_id': 'req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.559676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41edeacb10e44c678f18a00af8fbac2b', 'request_id': 'req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.586115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] The token cache is being invalidate because project 41edeacb10e44c678f18a00af8fbac2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:13.586300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.589014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.593141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 976/1925] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:13 2022] DELETE /identity/v3/projects/41edeacb10e44c678f18a00af8fbac2b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:13.597562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:13.614143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4ac-e7f8-44d7-b230-6ba011130be9 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:13.705702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.706885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.707070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.707423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] PATH_INFO: `/v3/users/edf658581d6c4e02a3b28b093cacb5ea` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.708810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf658581d6c4e02a3b28b093cacb5ea)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.712492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.738880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:13.745072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:13.766541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aab69-0cdb-41c1-90e9-74ff1ba85b10 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:13.781814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:13.782763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.783109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.783354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] PATH_INFO: `/v3/projects/a6725ed7d7e5494db17c117859d5c07c` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:13.786170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6725ed7d7e5494db17c117859d5c07c)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:13.790381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:13.828858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf658581d6c4e02a3b28b093cacb5ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.839737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf658581d6c4e02a3b28b093cacb5ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.841970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf658581d6c4e02a3b28b093cacb5ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.843083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf658581d6c4e02a3b28b093cacb5ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.888544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a98749d2-88c8-4c03-8770-dd5e9ffcaa33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf658581d6c4e02a3b28b093cacb5ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.922726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 950/1926] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:13 2022] DELETE /identity/v3/users/edf658581d6c4e02a3b28b093cacb5ea => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:13.929801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6725ed7d7e5494db17c117859d5c07c', 'request_id': 'req-9e135304-7d3c-4199-aa63-b658e9f70e01'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.945911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6725ed7d7e5494db17c117859d5c07c', 'request_id': 'req-9e135304-7d3c-4199-aa63-b658e9f70e01'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.954483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6725ed7d7e5494db17c117859d5c07c', 'request_id': 'req-9e135304-7d3c-4199-aa63-b658e9f70e01'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.978555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] The token cache is being invalidate because project a6725ed7d7e5494db17c117859d5c07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:13.978942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.979827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e135304-7d3c-4199-aa63-b658e9f70e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:13.981571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 977/1927] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:13 2022] DELETE /identity/v3/projects/a6725ed7d7e5494db17c117859d5c07c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:13.988919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57da3f-9338-406f-928e-df3be433026f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:13.989234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57da3f-9338-406f-928e-df3be433026f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:13.989686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57da3f-9338-406f-928e-df3be433026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.021298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5f57da3f-9338-406f-928e-df3be433026f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.023524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5f57da3f-9338-406f-928e-df3be433026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.117243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 951/1928] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:14.126647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.126902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.128214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.143568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57da3f-9338-406f-928e-df3be433026f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.148956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5f57da3f-9338-406f-928e-df3be433026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.161754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57da3f-9338-406f-928e-df3be433026f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.171270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.172511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.177364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.178172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.179015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.179300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] PATH_INFO: `/v3/projects/a479675e5da04e71aa05f96b99719fe8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.181510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a479675e5da04e71aa05f96b99719fe8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.184197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.193793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 978/1929] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:14.208909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.219667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba840e1c-6e73-46be-b7af-3dc05be75f89 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.231646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.232409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.237653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.237936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.238728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:14.239035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.242326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.273140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 979/1930] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:14.274284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a479675e5da04e71aa05f96b99719fe8', 'request_id': 'req-fc4830cd-623b-416b-9da2-be6909edaa34'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.279094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.288660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a479675e5da04e71aa05f96b99719fe8', 'request_id': 'req-fc4830cd-623b-416b-9da2-be6909edaa34'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.296179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9c202-5a2b-4b45-887b-144531232dcb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.297202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a479675e5da04e71aa05f96b99719fe8', 'request_id': 'req-fc4830cd-623b-416b-9da2-be6909edaa34'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.320205 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] The token cache is being invalidate because project a479675e5da04e71aa05f96b99719fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:14.320489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.328581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fc4830cd-623b-416b-9da2-be6909edaa34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.330593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 952/1931] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:14 2022] DELETE /identity/v3/projects/a479675e5da04e71aa05f96b99719fe8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:14.335019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.335307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.335585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.374955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.376099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.398279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.398650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.399569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.399993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.400848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.404762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.454907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 980/1932] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:14.463603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.469811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.485758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 953/1933] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:14.486649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.488582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.500734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1b110-1335-47cd-9d22-f397dbb0c9bb None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.556009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019521-88d5-47d2-971b-3d4b4f58e0f4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.562877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.563902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.564184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.564454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] PATH_INFO: `/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.566108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e929e0ef598e4d868a1e3df9ce134ee9)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.567670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.577674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.578004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.578286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.578589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.579290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.581441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee94de-db00-4d53-868b-6c51598da613 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.603220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 954/1934] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:14.605269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d20aa-14f3-4c3a-a190-9f489e909954 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.605269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d20aa-14f3-4c3a-a190-9f489e909954 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.605269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d20aa-14f3-4c3a-a190-9f489e909954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.623172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e929e0ef598e4d868a1e3df9ce134ee9', 'request_id': 'req-63a4f7e8-54d2-465a-99c2-471ebd427e8f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.623374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-249d20aa-14f3-4c3a-a190-9f489e909954 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.624759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-249d20aa-14f3-4c3a-a190-9f489e909954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.638944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e929e0ef598e4d868a1e3df9ce134ee9', 'request_id': 'req-63a4f7e8-54d2-465a-99c2-471ebd427e8f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.651236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e929e0ef598e4d868a1e3df9ce134ee9', 'request_id': 'req-63a4f7e8-54d2-465a-99c2-471ebd427e8f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.667425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] The token cache is being invalidate because project e929e0ef598e4d868a1e3df9ce134ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:14.667771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.668418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:14.670189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 981/1935] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:14 2022] DELETE /identity/v3/projects/e929e0ef598e4d868a1e3df9ce134ee9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:14.671275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.672641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.706974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 955/1936] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:14.708300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.708584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.708857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.729657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.730896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.739080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 956/1937] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:14.739878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.749552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99326076-908b-4b4d-9447-0a7e313c18e2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.756874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.757739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.758025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.758300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.759614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:14.759923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.761641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b62e7-3e8f-426e-9246-69cd4b213eaa admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.771296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f7e8-54d2-465a-99c2-471ebd427e8f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.781725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 957/1938] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:14.782646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.783380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.783636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.784721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.785346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.785460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.785824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.786848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.789963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.802944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 982/1939] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:14.803606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.812098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:14.813139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:14.815414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098896a-3413-46c1-8165-60eb10b5c3d2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.820442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.821630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.822220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.822332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 958/1940] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:14.822469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.822985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.824896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.826068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.835597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3083e7-16f4-4dcd-a201-34dcb4d89ce1 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.840898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.841571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.841851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.842123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.842885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.844477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.855355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 959/1941] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:14.858728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.869537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeebb-9fee-498c-a81d-b701f428f370 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:14.874852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:14.875620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:14.875807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:14.876113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:14.877417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:14.877417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:14.879168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:14.880958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 983/1942] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:14.882888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.898487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 960/1943] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:14.900621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:14.912307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be0553-6cbc-4cac-9a75-88b25c4afb5b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.000927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675f4f2-9253-4f39-aae4-8afe3d591936 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.019153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.020319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.020705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.021071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] PATH_INFO: `/v3/projects/83eb166d6c1241e49c45f56fbc0435ea/users/c035a97083ca433b8a58a08ac0a4770a/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.032722 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.033410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.033688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.033986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.034704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.036625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.054976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83eb166d6c1241e49c45f56fbc0435ea, user_id=c035a97083ca433b8a58a08ac0a4770a, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.059286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.069553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 961/1944] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:14 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:15.075217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.083753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 984/1945] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:14 2022] PUT /identity/v3/projects/83eb166d6c1241e49c45f56fbc0435ea/users/c035a97083ca433b8a58a08ac0a4770a/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:15.090016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a952-91e2-4eb5-af1b-56ff79e5c8e7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.105398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.187695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.188377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.188656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.188930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.189614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.191571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f42b7-a47f-4ed5-a0ad-fd530339e6ce admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.216122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0c6b1-8cde-4915-a8bb-cedf87811e8a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.222284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.223129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.224863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.224863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.224863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.226617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.236427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 962/1946] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:15.236427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.236427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.236427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.243715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 985/1947] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:15.243715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.270497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cee2b-f661-40a4-b7ee-56e58103b04e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.298689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] MFA Rules not processed for user `c035a97083ca433b8a58a08ac0a4770a`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:15.302096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:15.368535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.369224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.369566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.369854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.370671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.373042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.403802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 986/1948] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:15.403802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.411902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 963/1949] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:15.412807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.422162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32468cb2-9bc8-4142-973d-6e063e66fa61 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.425489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb66db-d3e7-418d-95ac-f5137013ffd8 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.427401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.428073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.428429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.428775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.429572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.431191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.431889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.432191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.432609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.433060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.433926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.436587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.448492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 987/1950] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:15.449487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.451314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:15.451936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 964/1951] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:15.452998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.467553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e8be-8b4c-4f69-bdf7-f841ac5161cc admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.476239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.477757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.478107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.478485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] PATH_INFO: `/v3/projects/5592661152b24accba1b0a108519dcde/users/4dcc55c90ec84e7f974e95bf183bc6eb/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.497243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5592661152b24accba1b0a108519dcde, user_id=4dcc55c90ec84e7f974e95bf183bc6eb, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.499963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.521636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 965/1952] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:15 2022] PUT /identity/v3/projects/5592661152b24accba1b0a108519dcde/users/4dcc55c90ec84e7f974e95bf183bc6eb/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:15.523011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.534947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fca14-81b1-4723-b798-233c6b511e70 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.596058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.596753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.597040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.597317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.600547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006f69-6805-49cd-8da9-ca7c2404f03f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.606349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.607261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.607691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.608173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.609085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.610780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c3079-48f0-491e-84ee-2b7b3c249656 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.626814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 966/1953] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:15.626814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.626814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.626814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.655055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] MFA Rules not processed for user `4dcc55c90ec84e7f974e95bf183bc6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:15.656183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:15.669182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.670639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.680088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 988/1954] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:15.681375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.706528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af5d76-5ff0-48ee-ac2d-12467583fda3 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.711473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.713106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.713783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.713997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] PATH_INFO: `/v3/projects/d9d6b5aeaee14d3a9b646cbefb1f2222/users/d54d538f62e64aeab667ee6d6867fd5b/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.733809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d6b5aeaee14d3a9b646cbefb1f2222, user_id=d54d538f62e64aeab667ee6d6867fd5b, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.736533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.742451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 967/1955] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:15.742451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.751991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78fc71-259e-40ae-a9a2-482723573bd2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.758763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 989/1956] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:15 2022] PUT /identity/v3/projects/d9d6b5aeaee14d3a9b646cbefb1f2222/users/d54d538f62e64aeab667ee6d6867fd5b/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:15.759825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:15.763636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:15.810785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.811660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.811945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.812228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] PATH_INFO: `/v3/users/c035a97083ca433b8a58a08ac0a4770a` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.813427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c035a97083ca433b8a58a08ac0a4770a)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.815402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.823075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506105f-3a73-4a42-89d8-30320a63fb65 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:15.826872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:15.827527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.828215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.828215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.865522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c035a97083ca433b8a58a08ac0a4770a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:15.873199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c035a97083ca433b8a58a08ac0a4770a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:15.874206 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c035a97083ca433b8a58a08ac0a4770a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:15.874937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c035a97083ca433b8a58a08ac0a4770a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:15.885358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d306f548-9424-4556-9d9a-8595730e63df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c035a97083ca433b8a58a08ac0a4770a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:15.896893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:15.898341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739653a-cfdc-4efc-b5d3-3228608eaa2b service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:15.904287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 968/1957] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:15 2022] DELETE /identity/v3/users/c035a97083ca433b8a58a08ac0a4770a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:15.906045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.906584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.907070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.911162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 990/1958] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:15.915916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:15.916259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:15.916539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:15.939069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:15.940529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:15.958729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] MFA Rules not processed for user `d54d538f62e64aeab667ee6d6867fd5b`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:15.960237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.026248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 991/1959] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:16.036243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.037896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.041137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 969/1960] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:16.045847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.047906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.065848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833328b6-a701-46d3-852b-4f365a009a43 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.071032 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.071748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.072033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.072307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.073243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.075216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.106892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 970/1961] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:16 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:16.111356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.112812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46acf5-9505-4631-bb53-78e1f486d7cc None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.163286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.163992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.164374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.165334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.178461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f67e3-ac4e-432d-803c-e85e2262ffb4 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.185759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.186465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.186756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.187041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.187745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.189985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.219025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 971/1962] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:16 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:16.221700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.232369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.234047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e1e5-e71f-4e90-aa23-ef755de7969c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.234453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.240176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.241471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.241782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.242070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.243278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.245066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.247920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 992/1963] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:16.258884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 972/1964] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:16.263460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.273887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c966b91-c81e-4889-b338-6dc27aae9c9c service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.279604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.280662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.280941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.281301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] PATH_INFO: `/v3/projects/376ca90b966d4bd0bd647d66b88c61da/users/ae0f30cd6f0b4ebd9e7cb662fa220cae/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.309908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376ca90b966d4bd0bd647d66b88c61da, user_id=ae0f30cd6f0b4ebd9e7cb662fa220cae, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.312739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.342568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 993/1965] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:16 2022] PUT /identity/v3/projects/376ca90b966d4bd0bd647d66b88c61da/users/ae0f30cd6f0b4ebd9e7cb662fa220cae/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:16.343217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.353416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe77b-6128-46e0-90cf-d9e3a135f967 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.390260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.393400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.425483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.426377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.426864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.427395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.428856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.430690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.445595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 973/1966] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:16.451029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.465769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4b04-39f6-47d8-8c16-9f64a6f9c987 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.480618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.482723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.484766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.485073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] PATH_INFO: `/v3/projects/376ca90b966d4bd0bd647d66b88c61da/users/ae0f30cd6f0b4ebd9e7cb662fa220cae/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.495652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376ca90b966d4bd0bd647d66b88c61da, user_id=ae0f30cd6f0b4ebd9e7cb662fa220cae, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.497498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53221f-4ad8-4e37-b9c7-9dfaccc61957 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.498837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae381-d1b0-4c3a-b7de-1d3255511d35 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.503138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.504134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.504506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.504869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] PATH_INFO: `/v3/users/c738eb2922d14b4b915ae1820c20fa42` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.506189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c738eb2922d14b4b915ae1820c20fa42)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.508865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.517679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 974/1967] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:16 2022] PUT /identity/v3/projects/376ca90b966d4bd0bd647d66b88c61da/users/ae0f30cd6f0b4ebd9e7cb662fa220cae/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:16.529718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.530083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.530461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.572908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c738eb2922d14b4b915ae1820c20fa42'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.573742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c738eb2922d14b4b915ae1820c20fa42'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.576418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] MFA Rules not processed for user `ae0f30cd6f0b4ebd9e7cb662fa220cae`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:16.577645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.581862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c738eb2922d14b4b915ae1820c20fa42'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.582810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c738eb2922d14b4b915ae1820c20fa42'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.593056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-accbbcda-fd96-47c3-8194-8823822ee58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c738eb2922d14b4b915ae1820c20fa42'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.616077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 994/1968] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:16 2022] DELETE /identity/v3/users/c738eb2922d14b4b915ae1820c20fa42 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:16.621459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.622164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.622608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.641568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:16.643237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.652999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 975/1969] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:16.666488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.670816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.725806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 995/1970] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:16.726771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.728078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.739164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516c6f7-e59a-4129-a3ac-3be70ee3fdda None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.739449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e2221-1326-4893-8fa2-7f87717a70d8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.744687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.745547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.745832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.746360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] PATH_INFO: `/v3/projects/83eb166d6c1241e49c45f56fbc0435ea` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.747621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83eb166d6c1241e49c45f56fbc0435ea)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.751677 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.788303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.788964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.789248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.789525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.809274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83eb166d6c1241e49c45f56fbc0435ea', 'request_id': 'req-abae1e86-9d7d-4b60-a71a-69d3f2f06959'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.819067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83eb166d6c1241e49c45f56fbc0435ea', 'request_id': 'req-abae1e86-9d7d-4b60-a71a-69d3f2f06959'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.825509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83eb166d6c1241e49c45f56fbc0435ea', 'request_id': 'req-abae1e86-9d7d-4b60-a71a-69d3f2f06959'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.838661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] The token cache is being invalidate because project 83eb166d6c1241e49c45f56fbc0435ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:16.839037 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.839685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:16.841248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 996/1971] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:16 2022] DELETE /identity/v3/projects/83eb166d6c1241e49c45f56fbc0435ea => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:16.845414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:16.846791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.863625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:16.865051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3355b-84ed-450f-b087-60169b67eca7 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:16.875937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 976/1972] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:16.885562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82297ef5-dcb6-4cc5-b459-b89957b6813c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.885853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82297ef5-dcb6-4cc5-b459-b89957b6813c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.886133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82297ef5-dcb6-4cc5-b459-b89957b6813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.909076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-82297ef5-dcb6-4cc5-b459-b89957b6813c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:16.910281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-82297ef5-dcb6-4cc5-b459-b89957b6813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:16.920952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1e86-9d7d-4b60-a71a-69d3f2f06959 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:16.926047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:16.926915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:16.927406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:16.927878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:16.930050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.010781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 977/1973] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:17.011433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.011433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.016844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.017214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.017543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.019712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 997/1974] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:17.057032 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:17.058653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.068253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 978/1975] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:17.072595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.086184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119fbaa-1eb4-4b1b-8ceb-422e0f39519d service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.091941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.092902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.093288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.098736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.099563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:17.099878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.103201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.131139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 998/1976] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:17.135112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.146019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672123e8-e5be-43e5-9160-c46cf6c845c4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.152727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.153473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.153830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.154112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.154994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.156980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.190624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 979/1977] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:17 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:17.194631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.210795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e066d-61e1-4b1b-acb1-6afcc198b2a9 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.277403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.278077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.278356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.279105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.279995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.281994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.308804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 999/1978] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:17 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:17.312980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.326167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75879-dd71-491d-ac2b-d75ceaff5133 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.330563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.331828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.332535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.332775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.333047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.333764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.335369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.337280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.346894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 980/1979] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:17.364585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.369794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43ee76-d866-4d76-9870-57a180e40f6d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.379709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.380958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.381343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.381692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] PATH_INFO: `/v3/users/d54d538f62e64aeab667ee6d6867fd5b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.383081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54d538f62e64aeab667ee6d6867fd5b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.384052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ebd58-9da7-4149-ab63-afdc327216d4 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.385699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.389082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.389755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.390038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.390317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.391069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.392665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.406296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 981/1980] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:17.408060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.410654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.427236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821e20-cdea-47ac-877b-b35c700fde48 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.432560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.433560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.433942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.434270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] PATH_INFO: `/v3/projects/0b7e984f320143279dcbad4d170c8c8b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.435527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7e984f320143279dcbad4d170c8c8b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.437636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.448325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54d538f62e64aeab667ee6d6867fd5b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.448984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54d538f62e64aeab667ee6d6867fd5b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.455843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54d538f62e64aeab667ee6d6867fd5b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.456514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54d538f62e64aeab667ee6d6867fd5b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.463449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54d538f62e64aeab667ee6d6867fd5b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.482820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1000/1981] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:17 2022] DELETE /identity/v3/users/d54d538f62e64aeab667ee6d6867fd5b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:17.487225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.488120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7e984f320143279dcbad4d170c8c8b', 'request_id': 'req-46e712a3-64a2-4001-85a9-22385fe0a9ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.488688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.494641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7e984f320143279dcbad4d170c8c8b', 'request_id': 'req-46e712a3-64a2-4001-85a9-22385fe0a9ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.504467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7e984f320143279dcbad4d170c8c8b', 'request_id': 'req-46e712a3-64a2-4001-85a9-22385fe0a9ea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.520219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] The token cache is being invalidate because project 0b7e984f320143279dcbad4d170c8c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:17.520861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.522506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-46e712a3-64a2-4001-85a9-22385fe0a9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.524964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 982/1982] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:17 2022] DELETE /identity/v3/projects/0b7e984f320143279dcbad4d170c8c8b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:17.526830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895bfc-1e79-476f-b3f5-e96f5a788db6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.527341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895bfc-1e79-476f-b3f5-e96f5a788db6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.528340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895bfc-1e79-476f-b3f5-e96f5a788db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.554022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3b895bfc-1e79-476f-b3f5-e96f5a788db6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:17.555958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3b895bfc-1e79-476f-b3f5-e96f5a788db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.581818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075568f-9e56-4db3-bb90-cf51f6fde74f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.586825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.587719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.588001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.588276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] PATH_INFO: `/v3/projects/f9fb651c72564bd0a8fe5090978fc50d/users/c0cd307eb71146e5a99f3ced3664b9ed/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.603258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fb651c72564bd0a8fe5090978fc50d, user_id=c0cd307eb71146e5a99f3ced3664b9ed, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.606080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.621949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1001/1983] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:17 2022] PUT /identity/v3/projects/f9fb651c72564bd0a8fe5090978fc50d/users/c0cd307eb71146e5a99f3ced3664b9ed/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:17.624384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 983/1984] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:17.627245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.631826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.635979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.636324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.636694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.672782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] MFA Rules not processed for user `c0cd307eb71146e5a99f3ced3664b9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:17.673989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.702772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf09b-05f9-4ef6-9a8e-4d0904022dc4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.707509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.708457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.708837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.709201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] PATH_INFO: `/v3/users/88442e67421b41dd9ea10e036b09e1c4` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.710688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88442e67421b41dd9ea10e036b09e1c4)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.713375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.740336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 984/1985] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:17.741290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.742801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.762576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88442e67421b41dd9ea10e036b09e1c4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.763911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88442e67421b41dd9ea10e036b09e1c4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.770837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88442e67421b41dd9ea10e036b09e1c4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.771504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88442e67421b41dd9ea10e036b09e1c4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.779711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88442e67421b41dd9ea10e036b09e1c4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:17.797650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1002/1986] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:17 2022] DELETE /identity/v3/users/88442e67421b41dd9ea10e036b09e1c4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:17.799380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:17.800819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.802642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84da202-4673-4e19-9b69-7a0d58afd752 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.856431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.857289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.857577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.857880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.860048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.870483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181ea4c-a171-4d09-8407-a40721a93f85 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:17.875548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:17.876288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:17.876553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:17.876826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:17.879024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:17.953836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.954896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.965765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 985/1987] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:17.970415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:17.971947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77d399-6e91-4910-83bd-dd7c21f9d045 service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:17.981584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1003/1988] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:18.141106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.142581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.172775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79345-094e-4774-831b-f8a1219e8570 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.184257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.185151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.185457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.185711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] PATH_INFO: `/v3/projects/bad828b2dd944740ac938dbf6373d8ea` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.191404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad828b2dd944740ac938dbf6373d8ea)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.193029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.251338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad828b2dd944740ac938dbf6373d8ea', 'request_id': 'req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.257313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad828b2dd944740ac938dbf6373d8ea', 'request_id': 'req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.263513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad828b2dd944740ac938dbf6373d8ea', 'request_id': 'req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.278092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] The token cache is being invalidate because project bad828b2dd944740ac938dbf6373d8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:18.278382 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.279059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.280995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 986/1989] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:18 2022] DELETE /identity/v3/projects/bad828b2dd944740ac938dbf6373d8ea => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:18.286363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b639e-28a5-4f48-b299-adf2acf664fa None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.286929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b639e-28a5-4f48-b299-adf2acf664fa None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.286929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b639e-28a5-4f48-b299-adf2acf664fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.306879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-696b639e-28a5-4f48-b299-adf2acf664fa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:18.310082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-696b639e-28a5-4f48-b299-adf2acf664fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.311518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.312604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.391250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1004/1990] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:18.392797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.393111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.393412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.410058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f1fd5-b890-4e4d-85c4-82d61f40003a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.415756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.416783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.417064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.417334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] PATH_INFO: `/v3/projects/d9d6b5aeaee14d3a9b646cbefb1f2222` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.418566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d6b5aeaee14d3a9b646cbefb1f2222)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.422440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.431544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:18.432782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.442949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1005/1991] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:18.448632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.458331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64630e-3a1b-4bdf-93fe-7c9174426c4d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.464859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.465707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.465990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.466263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.467080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:18.467388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.469381 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.493947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1006/1992] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:18.502328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.513436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d6b5aeaee14d3a9b646cbefb1f2222', 'request_id': 'req-8222d63f-0ba2-47ad-b1c3-becc53150b54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.513678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad03f9-5d6f-4474-801a-502ce485b7db admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.519865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d6b5aeaee14d3a9b646cbefb1f2222', 'request_id': 'req-8222d63f-0ba2-47ad-b1c3-becc53150b54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.529331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d6b5aeaee14d3a9b646cbefb1f2222', 'request_id': 'req-8222d63f-0ba2-47ad-b1c3-becc53150b54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.548346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] The token cache is being invalidate because project d9d6b5aeaee14d3a9b646cbefb1f2222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:18.548346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.548346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.549277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 987/1993] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:18 2022] DELETE /identity/v3/projects/d9d6b5aeaee14d3a9b646cbefb1f2222 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:18.553600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.555190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.582239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.583613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.584127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.585082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.586148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.588287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.621894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1007/1994] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:18 2022] POST /identity/v3/projects => generated 409 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:18.625616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.627241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.637030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222d63f-0ba2-47ad-b1c3-becc53150b54 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.705021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3216cd5-6b1b-4651-b35f-f33bb5bf39b6 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.711206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.712006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.712346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.712505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.712754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.713270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.713597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.713704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.713948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] PATH_INFO: `/v3/users/ae0f30cd6f0b4ebd9e7cb662fa220cae` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.715877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0f30cd6f0b4ebd9e7cb662fa220cae)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.716107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.718215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.748449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1008/1995] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:18 2022] POST /identity/v3/users => generated 305 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:18.755475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.769208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619705f-401e-47bb-883c-b1722f1d484e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.774879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.775621 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.775867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.776141 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.776881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.778784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.791863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1009/1996] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:18.792173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0f30cd6f0b4ebd9e7cb662fa220cae'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.795521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.802015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0f30cd6f0b4ebd9e7cb662fa220cae'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.803373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0f30cd6f0b4ebd9e7cb662fa220cae'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.804927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0f30cd6f0b4ebd9e7cb662fa220cae'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.810295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f2f37-7e15-4b35-a9db-1f8ae1870e3f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.816701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7e9414ca-5649-479b-bd55-c63e642a10a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0f30cd6f0b4ebd9e7cb662fa220cae'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:18.818452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.819150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.819480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.819757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.820556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.824801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.837302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1010/1997] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:18.844046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:18.847684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:18.849208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 988/1998] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:18 2022] DELETE /identity/v3/users/ae0f30cd6f0b4ebd9e7cb662fa220cae => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:18.869365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce775d-42bf-4ac8-87de-4dc7409fc4b9 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:18.934803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:18.935704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.935991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.936275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] PATH_INFO: `/v3/projects/213406e432404a99bc515c1728f06b1e/users/8a9cf7ff5d0c4abb898fba45a8718520/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:18.951563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213406e432404a99bc515c1728f06b1e, user_id=8a9cf7ff5d0c4abb898fba45a8718520, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:18.954216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:18.970932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1011/1999] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:18 2022] PUT /identity/v3/projects/213406e432404a99bc515c1728f06b1e/users/8a9cf7ff5d0c4abb898fba45a8718520/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:18.982922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:18.983214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:18.983488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.019093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] MFA Rules not processed for user `8a9cf7ff5d0c4abb898fba45a8718520`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:19.020571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.091776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 989/2000] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:19.096318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.143293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.144712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.157938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7223904-62df-409b-b235-05f7657afb52 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.170086 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.170971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.171224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.173335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.174093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.176087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.206012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1012/2001] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/projects => generated 409 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:19.207024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.219113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74685f-4703-4f6d-ade7-62841750b753 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.279627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6aaf-ec70-44ec-9e3f-504f7b6d3664 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.281407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.282300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.282623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.282897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] PATH_INFO: `/v3/projects/376ca90b966d4bd0bd647d66b88c61da` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.284072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376ca90b966d4bd0bd647d66b88c61da)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.284250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.284884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.285168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.285443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.286028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.286136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.288045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.308194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1013/2002] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/users => generated 311 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:19.312346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.322499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeb474-bfd1-45d7-b9d0-8112978e0edb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.331714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376ca90b966d4bd0bd647d66b88c61da', 'request_id': 'req-efda57c1-ef8a-4bfc-b3a9-2e77269de136'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:19.337501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376ca90b966d4bd0bd647d66b88c61da', 'request_id': 'req-efda57c1-ef8a-4bfc-b3a9-2e77269de136'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:19.345331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376ca90b966d4bd0bd647d66b88c61da', 'request_id': 'req-efda57c1-ef8a-4bfc-b3a9-2e77269de136'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:19.359982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] The token cache is being invalidate because project 376ca90b966d4bd0bd647d66b88c61da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:19.360424 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:19.361132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efda57c1-ef8a-4bfc-b3a9-2e77269de136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:19.362727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 990/2003] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:19 2022] DELETE /identity/v3/projects/376ca90b966d4bd0bd647d66b88c61da => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:19.367457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.367835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.368112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.387132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.388283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.388589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.388857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.389584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.390002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:19.391269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.391420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10393ca9-f80b-43e4-b5e2-53c5422da2b5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.402161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 991/2004] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:19.405414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1014/2005] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:19.409455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.410836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.410956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.411176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.411454 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.422484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4960a8d-2594-481d-b5ae-bd58b0d40d7b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.427151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.428055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.428338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.428611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] PATH_INFO: `/v3/projects/4dface454a7547b0846f78356e1467f1/users/90a41c9d4f45421292b94a45fdd6e4d8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.430837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:19.432012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.440607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1015/2006] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:19.443052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dface454a7547b0846f78356e1467f1, user_id=90a41c9d4f45421292b94a45fdd6e4d8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.444465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.446273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.455291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c89b9-4ffb-4602-83cc-2656c8840d3d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.466508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 992/2007] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:19 2022] PUT /identity/v3/projects/4dface454a7547b0846f78356e1467f1/users/90a41c9d4f45421292b94a45fdd6e4d8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:19.468180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.510699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.511148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.511429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.511771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.512393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:19.512695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.514570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.529427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1016/2008] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:19.533197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.535938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25806f-7ede-4d92-8b00-cd7e7c12f4e5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.546213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.546995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.547280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.547559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.548627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.550052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52005b8-04c0-41b2-88a8-a800d00d2946 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.550181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.561055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.561948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.562300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.562695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.563750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.566235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.573491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 993/2009] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:19.580079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.592232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5885-f3c9-4e2e-8ebd-8d3baf3c837c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.595782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1017/2010] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:19.598050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.599179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.599495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.599748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] PATH_INFO: `/v3/projects/4dface454a7547b0846f78356e1467f1/users/90a41c9d4f45421292b94a45fdd6e4d8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.600087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.601723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dface454a7547b0846f78356e1467f1, user_id=90a41c9d4f45421292b94a45fdd6e4d8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.605186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eb9a5-2368-485c-8cab-373b6aaa57f6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.616320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 994/2011] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:19 2022] PUT /identity/v3/projects/4dface454a7547b0846f78356e1467f1/users/90a41c9d4f45421292b94a45fdd6e4d8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:19.631289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.631923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.631923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.673572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] MFA Rules not processed for user `90a41c9d4f45421292b94a45fdd6e4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:19.674544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807695-2088-4382-b421-b1d48b762c46 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.675956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.739359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.740087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.740367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.740638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.741325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.743350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.759371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 995/2012] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:19.770178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.771609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:19.773142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1018/2013] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:19.774447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.796309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292835d-3032-4370-95e9-096cf86cd548 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.802478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.803151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.803437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.803718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.804445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.813797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.827887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1019/2014] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:19.828743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.838734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b48fc8-f550-41eb-9b4a-93537e0bdcaa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.840297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce90d9-430b-44d9-b7c8-285d0f4165f4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.843383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.843810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.844030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.844314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.844458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.844614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.844719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.844972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.845282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.846990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.854963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.856393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d22b5-a1b0-423e-8f58-79edaff73d66 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.863016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1020/2015] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:19.867055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:19.872114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 996/2016] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:19.877124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff171-1be4-43d4-904a-8ef95994e6aa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:19.881952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:19.882862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.883140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.883447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] PATH_INFO: `/v3/projects/c6c1855f54374c9d8b4f10163686898f/users/303d2fd8836d4c689ccdb4b4b8c09a73/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:19.932686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c1855f54374c9d8b4f10163686898f, user_id=303d2fd8836d4c689ccdb4b4b8c09a73, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:19.935947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:19.963727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1021/2017] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:19 2022] PUT /identity/v3/projects/c6c1855f54374c9d8b4f10163686898f/users/303d2fd8836d4c689ccdb4b4b8c09a73/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:19.972533 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:19.972847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:19.973097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.018302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] MFA Rules not processed for user `303d2fd8836d4c689ccdb4b4b8c09a73`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:20.019848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:20.088024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.101044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 997/2018] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:20.105706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.145657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f71ff-f538-4071-b348-368f77c6d140 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.150001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.151081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.151406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.151696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.156496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:20.170830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76eb4-77cd-4fb7-a09b-cdbf78756dc9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.176178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.176773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.177565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.177862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.178996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.181100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.209003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 998/2019] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:20 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:20.212835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.238254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.239887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.249711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1022/2020] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:20.276246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfbe65-a9d4-47de-b1af-f471c2f85be2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.281196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.281912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.282135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.282432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.283124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.285010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.304803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 999/2021] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:20 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:20.309665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.320109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc71a09-3c23-4aee-965f-203562d98657 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.325537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.326228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.326777 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.327183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.328048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.329650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.343456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1023/2022] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:20.344626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.354973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf27c3-8ea8-4b53-8da5-d2fa552d1a05 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.359659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.360551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.360835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.361122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] PATH_INFO: `/v3/projects/4ddc66c388ce4137ae60b91a209dc266/users/045700cac1024dc2ab6e96394337b4f6/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.379014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddc66c388ce4137ae60b91a209dc266, user_id=045700cac1024dc2ab6e96394337b4f6, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.381713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.394125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1000/2023] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:20 2022] PUT /identity/v3/projects/4ddc66c388ce4137ae60b91a209dc266/users/045700cac1024dc2ab6e96394337b4f6/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:20.398189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.410113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45d5d-ab52-47c3-bfc0-8adf50dc3552 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.467070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.467738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.468019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.468293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.469004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.470641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867fba8-477d-4f5a-b0f1-df73da27c48a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.484042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1024/2024] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:20.485647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.496553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16ad0-119a-47fb-b758-d2a6731960b5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.501733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.502738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.503021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.503281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] PATH_INFO: `/v3/projects/4ddc66c388ce4137ae60b91a209dc266/users/045700cac1024dc2ab6e96394337b4f6/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.505541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddc66c388ce4137ae60b91a209dc266, user_id=045700cac1024dc2ab6e96394337b4f6, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.508385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.519505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1001/2025] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:20 2022] PUT /identity/v3/projects/4ddc66c388ce4137ae60b91a209dc266/users/045700cac1024dc2ab6e96394337b4f6/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:20.531603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.532094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.532541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.560317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] MFA Rules not processed for user `045700cac1024dc2ab6e96394337b4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:20.561433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:20.633133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1025/2026] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:20.645604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:20.647317 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:20.704104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318a010-0139-455c-ab7a-b857c99d4f76 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:20.708011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:20.708772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:20.709061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:20.709333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:20.718765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:20.720180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:20.730509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1002/2027] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:21.041980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.051807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bde86b-f67b-4ab1-8dd7-408b5b5b050c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.056410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045700cac1024dc2ab6e96394337b4f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ddc66c388ce4137ae60b91a209dc266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.057056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.057338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.057612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.058354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:21.058690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.060250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.072499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1026/2028] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:21.076588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.085633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7fb78-11a6-4859-b3a5-7895f903d246 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.091015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045700cac1024dc2ab6e96394337b4f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ddc66c388ce4137ae60b91a209dc266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.091666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.091944 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.092337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.092905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.094869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.119861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1003/2029] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:21 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:21.124171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.134477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d8844-9cc2-4b59-95b4-a8b84e4543b6 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.194888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045700cac1024dc2ab6e96394337b4f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ddc66c388ce4137ae60b91a209dc266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.195552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.196409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.196682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.197398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:21.197701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.199364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.214229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1027/2030] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:21.247399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.258620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f8260-1cb8-49bc-b24c-9a470d61c130 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.263384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045700cac1024dc2ab6e96394337b4f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ddc66c388ce4137ae60b91a209dc266', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.264322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.264672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.265044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] PATH_INFO: `/v3/projects/1e44e6e00fdf4f27a8c7060c63f0e91d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.266285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1e44e6e00fdf4f27a8c7060c63f0e91d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.268306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.317906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e44e6e00fdf4f27a8c7060c63f0e91d', 'request_id': 'req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.323291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e44e6e00fdf4f27a8c7060c63f0e91d', 'request_id': 'req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.330009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e44e6e00fdf4f27a8c7060c63f0e91d', 'request_id': 'req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.343108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] The token cache is being invalidate because project 1e44e6e00fdf4f27a8c7060c63f0e91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:21.343295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.343947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0505df1c-1e3d-48d1-b0ef-4f52ffba86e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.346155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1004/2031] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:21 2022] DELETE /identity/v3/projects/1e44e6e00fdf4f27a8c7060c63f0e91d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:21.540772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.542572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:21.607634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ee01-e0d3-4f96-8e72-86dedb8454e7 tempest-VolumeQuotaClassesTest-110999950 tempest-VolumeQuotaClassesTest-110999950-project-admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.671862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.672812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.673183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.673514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] PATH_INFO: `/v3/users/303d2fd8836d4c689ccdb4b4b8c09a73` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.674813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303d2fd8836d4c689ccdb4b4b8c09a73)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.676803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.730653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303d2fd8836d4c689ccdb4b4b8c09a73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.731312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303d2fd8836d4c689ccdb4b4b8c09a73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.739453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303d2fd8836d4c689ccdb4b4b8c09a73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.740208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303d2fd8836d4c689ccdb4b4b8c09a73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.751872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303d2fd8836d4c689ccdb4b4b8c09a73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:21.772826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1028/2032] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:21 2022] DELETE /identity/v3/users/303d2fd8836d4c689ccdb4b4b8c09a73 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:21.777863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.778151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.778522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.795968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:21.797145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:21.854829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1005/2033] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:21.864462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:21.865945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:21.925083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3dd2b-34dd-41eb-bcd7-c196ffe80f05 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:21.929851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:21.930564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:21.930840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:21.931121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:21.940568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:21.942024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:21.951541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1029/2034] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:22.215029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:22.216515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:22.231302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3869e-6b4b-40cf-b386-6e70fa88267b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:22.236664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:22.237612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:22.238120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:22.238538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] PATH_INFO: `/v3/projects/c6c1855f54374c9d8b4f10163686898f` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:22.240076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c1855f54374c9d8b4f10163686898f)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:22.243275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:22.291065 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c1855f54374c9d8b4f10163686898f', 'request_id': 'req-2c1252c9-51f4-4663-927a-fa0eecf43112'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.296710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c1855f54374c9d8b4f10163686898f', 'request_id': 'req-2c1252c9-51f4-4663-927a-fa0eecf43112'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.302844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c1855f54374c9d8b4f10163686898f', 'request_id': 'req-2c1252c9-51f4-4663-927a-fa0eecf43112'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.316067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] The token cache is being invalidate because project c6c1855f54374c9d8b4f10163686898f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:22.316358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.316987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.318573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1006/2035] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:22 2022] DELETE /identity/v3/projects/c6c1855f54374c9d8b4f10163686898f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:22.323497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:22.333998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515a2e3-4fb5-453d-b5e5-160cbe1d2826 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:22.408847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:22.409681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:22.409919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:22.410191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] PATH_INFO: `/v3/users/045700cac1024dc2ab6e96394337b4f6` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:22.411402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=045700cac1024dc2ab6e96394337b4f6)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:22.413317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:22.459770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045700cac1024dc2ab6e96394337b4f6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.460417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045700cac1024dc2ab6e96394337b4f6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.468069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045700cac1024dc2ab6e96394337b4f6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.468736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045700cac1024dc2ab6e96394337b4f6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.476032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b72ae6b0-a538-4bf9-af71-28f3c011efa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045700cac1024dc2ab6e96394337b4f6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.493680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1030/2036] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:22 2022] DELETE /identity/v3/users/045700cac1024dc2ab6e96394337b4f6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:22.760927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:22.762530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:22.775711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1252c9-51f4-4663-927a-fa0eecf43112 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:22.840020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:22.840883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:22.841166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:22.841438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] PATH_INFO: `/v3/projects/4ddc66c388ce4137ae60b91a209dc266` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:22.842642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddc66c388ce4137ae60b91a209dc266)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:22.844543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:22.896989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddc66c388ce4137ae60b91a209dc266', 'request_id': 'req-e815aaf2-50ac-4ce8-bbf3-019ef5402767'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.902882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddc66c388ce4137ae60b91a209dc266', 'request_id': 'req-e815aaf2-50ac-4ce8-bbf3-019ef5402767'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.909265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddc66c388ce4137ae60b91a209dc266', 'request_id': 'req-e815aaf2-50ac-4ce8-bbf3-019ef5402767'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.923889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] The token cache is being invalidate because project 4ddc66c388ce4137ae60b91a209dc266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:22.924385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.925031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e815aaf2-50ac-4ce8-bbf3-019ef5402767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:22.926596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1007/2037] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:22 2022] DELETE /identity/v3/projects/4ddc66c388ce4137ae60b91a209dc266 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:22.931998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:22.932740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:22.932740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:22.953164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:22.954432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:23.012792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1031/2038] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:23.020351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.020644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.020950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.040100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:23.041278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:23.049027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1008/2039] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:23.053178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.063458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139895-cfc5-45c3-a723-d531098fe927 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.068220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.068864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.069147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.069421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.070138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:23.070474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.072373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.083561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1032/2040] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:23.087877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.096964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cedc-10b6-4fd6-8e5a-c26ac61af604 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.101503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.102693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.102972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.103285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.104010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.105891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.131091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1009/2041] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/projects => generated 401 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:23.135757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.145512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d21f3-eded-4ac8-a28a-5f3f957c5051 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.197711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.198315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.198624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.198900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.199584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.201473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.221267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1033/2042] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/users => generated 301 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:23.225931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.236121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b1bd-06cb-4311-9456-43aa58f87c55 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.240611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.241261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.241538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.241808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.242576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.244156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.254333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1010/2043] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:23.258916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.268942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbeceb-eaf4-4a98-a6c4-f281e5bfdcba admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.273488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.274785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.275070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.275347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.276065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.277637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a26720-1634-47bf-8731-6c843822b280 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.287282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1034/2044] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:23.290997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.300463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359724-fecd-4fb0-8b18-314af2024ac1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.305255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.306153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.306464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.306858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] PATH_INFO: `/v3/projects/1f5db11520aa494daf6c0ea51416d5e5/users/0fe9f26d7e8749db9b15a00b7814501c/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.323925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5db11520aa494daf6c0ea51416d5e5, user_id=0fe9f26d7e8749db9b15a00b7814501c, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.326644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.340401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1011/2045] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:23 2022] PUT /identity/v3/projects/1f5db11520aa494daf6c0ea51416d5e5/users/0fe9f26d7e8749db9b15a00b7814501c/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:23.350617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.350906 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.351182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.378348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] MFA Rules not processed for user `0fe9f26d7e8749db9b15a00b7814501c`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:23.379570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:23.449735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1035/2046] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:23.450955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.511085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7ca5-2e0a-44db-813d-bcd986b3c55e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.515745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.516399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.516686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.525898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.525898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.525898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.547015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1012/2047] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:23.550900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.562503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b3f49-2345-4246-8421-2cae3cd54d8e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.618270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.619050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.619277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.619579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.620366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.622650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.649199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1036/2048] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/users => generated 306 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:23.650841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.662510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75bd9-9344-4add-8297-f73e3c7df6ba admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.667350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.668086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.668367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.668644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.669502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.671180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.683058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1013/2049] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:23.687263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.697816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7402888-1fee-433f-b91f-0f40e63bee06 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.702831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.703646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.703922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.704189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] PATH_INFO: `/v3/projects/137c096c7b3b4212adf4953db5e88673/users/7e7d02e589674d50b0ae59e39d47936e/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.720409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137c096c7b3b4212adf4953db5e88673, user_id=7e7d02e589674d50b0ae59e39d47936e, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.723071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.743803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1037/2050] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:23 2022] PUT /identity/v3/projects/137c096c7b3b4212adf4953db5e88673/users/7e7d02e589674d50b0ae59e39d47936e/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:23.748053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.758346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4e22-6a90-4bb8-bbbe-d38f5f7ec4e6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.812678 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.813343 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.813624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.813896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.814681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.816223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa9014-57ff-4224-9535-48f4879b53ff admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.826172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1014/2051] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:23.830323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.841609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca006be-92d0-41d7-a671-1510ffbc4441 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:23.846507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:23.847562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.847974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.848315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] PATH_INFO: `/v3/projects/137c096c7b3b4212adf4953db5e88673/users/7e7d02e589674d50b0ae59e39d47936e/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.850492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=137c096c7b3b4212adf4953db5e88673, user_id=7e7d02e589674d50b0ae59e39d47936e, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:23.855662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:23.864341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1038/2052] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:23 2022] PUT /identity/v3/projects/137c096c7b3b4212adf4953db5e88673/users/7e7d02e589674d50b0ae59e39d47936e/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:23.876264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:23.876556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:23.876838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:23.901842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] MFA Rules not processed for user `7e7d02e589674d50b0ae59e39d47936e`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:23.903064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:23.969705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1015/2053] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:23.980929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:23.982767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:24.038173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971261-a310-4e50-8452-88b3490cf7ab admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:24.042400 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:24.043088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:24.043370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:24.043645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:24.053097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:24.054576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:24.065903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1039/2054] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:24.140650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:24.150537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc560d30-6b21-4d84-8a8a-d97207ae86b6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:24.154110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:24.154779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:24.155061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:24.155335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:24.210740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:24.212724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea898cf-fa2c-42f5-837f-d7714bcb7bb6 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:24.221745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1016/2055] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:29.672252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:29.673710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:29.740146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19baf50e-51e3-4c14-97cb-26401d22c594 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:29.744559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:29.745422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:29.745902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:29.746514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] PATH_INFO: `/v3/users/8a9cf7ff5d0c4abb898fba45a8718520` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:29.747736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9cf7ff5d0c4abb898fba45a8718520)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:29.749656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:29.794302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9cf7ff5d0c4abb898fba45a8718520'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:29.794988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9cf7ff5d0c4abb898fba45a8718520'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:29.801512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9cf7ff5d0c4abb898fba45a8718520'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:29.802188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9cf7ff5d0c4abb898fba45a8718520'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:29.809661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9cf7ff5d0c4abb898fba45a8718520'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:29.827767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1040/2056] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:29 2022] DELETE /identity/v3/users/8a9cf7ff5d0c4abb898fba45a8718520 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:29.832315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:29.832850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:29.832850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:29.850529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:29.851762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:29.910018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1017/2057] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:29.919631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:29.921101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:29.976324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06bcbb-585e-4c8e-9b15-e93309b61f0c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:29.980444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:29.981333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:29.981598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:29.981889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:29.991724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:29.993074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:30.002505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1041/2058] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:30.270658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:30.272100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.286734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d9e2-fbe0-4f10-b3f0-e2b090172bd2 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:30.303002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:30.303913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.304197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.304474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] PATH_INFO: `/v3/projects/213406e432404a99bc515c1728f06b1e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.306639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213406e432404a99bc515c1728f06b1e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:30.308558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:30.369274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213406e432404a99bc515c1728f06b1e', 'request_id': 'req-4f65d32f-52f1-4061-89cb-c27e428fd9f7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.378619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213406e432404a99bc515c1728f06b1e', 'request_id': 'req-4f65d32f-52f1-4061-89cb-c27e428fd9f7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.386269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213406e432404a99bc515c1728f06b1e', 'request_id': 'req-4f65d32f-52f1-4061-89cb-c27e428fd9f7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.402910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:30.402910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.409759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] The token cache is being invalidate because project 213406e432404a99bc515c1728f06b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:30.409998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a3f1-afeb-427c-b6f4-702940e0ef70 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:30.410096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.411059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.413223 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1018/2059] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:30 2022] DELETE /identity/v3/projects/213406e432404a99bc515c1728f06b1e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:30.419167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:30.420841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.478666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:30.479638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.480109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.480493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] PATH_INFO: `/v3/users/0fe9f26d7e8749db9b15a00b7814501c` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.481763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe9f26d7e8749db9b15a00b7814501c)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:30.483811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:30.486607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d32f-52f1-4061-89cb-c27e428fd9f7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:30.491567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:30.494046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.494046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.494046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] PATH_INFO: `/v3/users/90a41c9d4f45421292b94a45fdd6e4d8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.496870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a41c9d4f45421292b94a45fdd6e4d8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:30.498674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:30.539217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe9f26d7e8749db9b15a00b7814501c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.540584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe9f26d7e8749db9b15a00b7814501c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.547810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe9f26d7e8749db9b15a00b7814501c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.548477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe9f26d7e8749db9b15a00b7814501c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.552633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a41c9d4f45421292b94a45fdd6e4d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.559966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a41c9d4f45421292b94a45fdd6e4d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.560880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a41c9d4f45421292b94a45fdd6e4d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.562157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a41c9d4f45421292b94a45fdd6e4d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.567057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d97701ea-2063-4683-8455-159d2b89e908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe9f26d7e8749db9b15a00b7814501c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.571325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a41c9d4f45421292b94a45fdd6e4d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:30.593003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1042/2060] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:30 2022] DELETE /identity/v3/users/0fe9f26d7e8749db9b15a00b7814501c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:30.594189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1019/2061] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:30 2022] DELETE /identity/v3/users/90a41c9d4f45421292b94a45fdd6e4d8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:30.597020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.598166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.598776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.619178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:30.621591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.710912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1043/2062] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:30.721739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:30.723372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.795155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45d9c-4069-4e0b-90b5-3912664812ef admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:30.799167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:30.799881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.800232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.800509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.809753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:30.811241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef99d5a-813e-449d-b4bf-f1c97f2a0ca4 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:30.821449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1020/2063] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:30.940911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:30.942838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:30.955676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd16bf-a882-424e-8456-dcf6d44caa16 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:30.961012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:30.962580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:30.962883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:30.963131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] PATH_INFO: `/v3/projects/4dface454a7547b0846f78356e1467f1` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:30.964395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dface454a7547b0846f78356e1467f1)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:30.966350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.026277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dface454a7547b0846f78356e1467f1', 'request_id': 'req-4936d234-29a8-4730-b2bb-495892843861'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.032789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dface454a7547b0846f78356e1467f1', 'request_id': 'req-4936d234-29a8-4730-b2bb-495892843861'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.038704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dface454a7547b0846f78356e1467f1', 'request_id': 'req-4936d234-29a8-4730-b2bb-495892843861'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.055286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] The token cache is being invalidate because project 4dface454a7547b0846f78356e1467f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:31.055583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.056336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-4936d234-29a8-4730-b2bb-495892843861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.057886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1044/2064] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:30 2022] DELETE /identity/v3/projects/4dface454a7547b0846f78356e1467f1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:31.063527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.063997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.064094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.082304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:31.083627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.151751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1021/2065] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:31.161331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.161569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.161847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.182001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:31.183269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.185937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.187286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.193733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1045/2066] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:31.199881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.206456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137389b-0411-4f1a-87bf-935cdd964735 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.213287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29eee-b807-44fe-810f-7816e02d0168 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.218026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.219056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.219247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.219680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.219982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.220558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] PATH_INFO: `/v3/projects/1f5db11520aa494daf6c0ea51416d5e5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.220666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.220666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.221341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:31.221688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.221794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5db11520aa494daf6c0ea51416d5e5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.223441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.223916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.241347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1046/2067] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:31.242225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.252324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b448dd-bba3-4b86-9efa-17da234067b1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.257853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.259095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.259834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.259834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.260650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.262312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.275265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5db11520aa494daf6c0ea51416d5e5', 'request_id': 'req-efbb1dc3-78ef-4779-b312-13aead85ccf8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.281995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5db11520aa494daf6c0ea51416d5e5', 'request_id': 'req-efbb1dc3-78ef-4779-b312-13aead85ccf8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.291187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1047/2068] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:31.292445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5db11520aa494daf6c0ea51416d5e5', 'request_id': 'req-efbb1dc3-78ef-4779-b312-13aead85ccf8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.294485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.306853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] The token cache is being invalidate because project 1f5db11520aa494daf6c0ea51416d5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:31.307172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.307816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.309381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1022/2069] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:31 2022] DELETE /identity/v3/projects/1f5db11520aa494daf6c0ea51416d5e5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:31.313109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.314532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.357603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397974-e8fb-41cf-9cb3-1758eca861c3 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.362192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.362866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.363152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.363427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.364105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.366118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.377271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb1dc3-78ef-4779-b312-13aead85ccf8 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.382651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.384051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.384335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.384608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] PATH_INFO: `/v3/users/7e7d02e589674d50b0ae59e39d47936e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.385819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7d02e589674d50b0ae59e39d47936e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.387788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.389981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1048/2070] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:31.393729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.395374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.407956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dee70-b183-41e2-846d-14205f7ff745 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.412608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.413276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.413561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.413833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.414620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.416403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.430299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1049/2071] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:31.431396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.437831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7d02e589674d50b0ae59e39d47936e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.440892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3b17c-2e8a-4004-b0c9-543e78be7aa4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.444430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7d02e589674d50b0ae59e39d47936e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.445106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7d02e589674d50b0ae59e39d47936e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.445957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.446336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7d02e589674d50b0ae59e39d47936e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.446618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.446913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.447169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.447900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.449462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.453650 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b8b9dc38-7b8e-475e-963a-8b25dafdb697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7d02e589674d50b0ae59e39d47936e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.462143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1050/2072] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:31.466210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.467870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.472597 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1023/2073] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:31 2022] DELETE /identity/v3/users/7e7d02e589674d50b0ae59e39d47936e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:31.480488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72bd8-2bc9-4ca8-9930-629f104211f1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.536520 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.537511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.537930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.538314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] PATH_INFO: `/v3/projects/713227acbbb545acbbaf9fc35efd72a9/users/4a86f0e32a0f4a43be6d2e955845d3c2/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.554326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=713227acbbb545acbbaf9fc35efd72a9, user_id=4a86f0e32a0f4a43be6d2e955845d3c2, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.557509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.571892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1051/2074] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:31 2022] PUT /identity/v3/projects/713227acbbb545acbbaf9fc35efd72a9/users/4a86f0e32a0f4a43be6d2e955845d3c2/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:31.582679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.583047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.583389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.610238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] MFA Rules not processed for user `4a86f0e32a0f4a43be6d2e955845d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:31.611585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.682274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1024/2075] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:31.686792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.747644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.749333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.751962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e50d-5435-4d63-8866-a7d7d36cbd8c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.758578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.759250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.759539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.759818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.760577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.765667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.766478 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216a179-e7f5-4379-9d2f-b1627428c6d7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.775438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.776326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.776609 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.776881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] PATH_INFO: `/v3/projects/137c096c7b3b4212adf4953db5e88673` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.778111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137c096c7b3b4212adf4953db5e88673)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.781492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.801186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1052/2076] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:31.804612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.845608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137c096c7b3b4212adf4953db5e88673', 'request_id': 'req-fb0169ea-f103-4680-87e3-bc40479746a6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.858730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137c096c7b3b4212adf4953db5e88673', 'request_id': 'req-fb0169ea-f103-4680-87e3-bc40479746a6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.865054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137c096c7b3b4212adf4953db5e88673', 'request_id': 'req-fb0169ea-f103-4680-87e3-bc40479746a6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.879724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ada4e-e9c6-4840-aa53-671c5616cab5 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:31.880118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] The token cache is being invalidate because project 137c096c7b3b4212adf4953db5e88673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:31.880522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.881301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fb0169ea-f103-4680-87e3-bc40479746a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:31.883175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1025/2077] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:31 2022] DELETE /identity/v3/projects/137c096c7b3b4212adf4953db5e88673 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:31.888172 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eec7d-ed33-4fa3-a682-f9f2c6a421a0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.889314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eec7d-ed33-4fa3-a682-f9f2c6a421a0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.889903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eec7d-ed33-4fa3-a682-f9f2c6a421a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.910351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e10eec7d-ed33-4fa3-a682-f9f2c6a421a0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:31.911586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e10eec7d-ed33-4fa3-a682-f9f2c6a421a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.947805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:31.948567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:31.948913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:31.949287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:31.950053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:31.952073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:31.974607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1053/2078] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:31.979041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:31.980888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:31.993191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fccb32-33da-4b62-b57e-606deb91e9fc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.000485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1026/2079] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:32.003530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86a23-1b52-4032-8277-63761160e502 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.003530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86a23-1b52-4032-8277-63761160e502 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.003530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86a23-1b52-4032-8277-63761160e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.004863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.005806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.006140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.006367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.007125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.009339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.020762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1054/2080] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:32.025774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-15f86a23-1b52-4032-8277-63761160e502 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:32.026735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.027007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-15f86a23-1b52-4032-8277-63761160e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.036338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1027/2081] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:32.037195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9aafb-7010-49f5-8561-ac3ecbd360d5 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.040428 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f86a23-1b52-4032-8277-63761160e502 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.042908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.043789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.044069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.044343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] PATH_INFO: `/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535/users/729fdfabb5ad4592b04c14910a05ebeb/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.050579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f86a23-1b52-4032-8277-63761160e502 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.056267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.057051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.057437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.057717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.058489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:32.058783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.060449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.065263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac64954f6e141d5b6e2cad4e2a0e535, user_id=729fdfabb5ad4592b04c14910a05ebeb, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.071583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.077163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1028/2082] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:32.078086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.093593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc401ea-f8e0-4357-bd96-2b7c38950e95 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.094712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1055/2083] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:32 2022] PUT /identity/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535/users/729fdfabb5ad4592b04c14910a05ebeb/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:32.098879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.160274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.161306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.161607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.162021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.162851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.167536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.188480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b00fd-d2e2-4aad-bbe3-bffce388d225 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.193914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.195007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.195873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.196297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.197249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.199221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.217325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1056/2084] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:32.218160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.218305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1029/2085] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:32.223296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.231746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb37ed-4f88-4074-bfa8-35e11fe2b489 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.297209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.298091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.298487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.298767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.299468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.302147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac84e-72b6-4c59-9968-268109636822 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.302715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055a078-f0bf-4544-9533-cf885793993b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.307744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aff51-533c-447a-8f19-219120730914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.308767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aff51-533c-447a-8f19-219120730914 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.309049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aff51-533c-447a-8f19-219120730914 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.309322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aff51-533c-447a-8f19-219120730914 admin admin] PATH_INFO: `/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535/users/729fdfabb5ad4592b04c14910a05ebeb/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.311365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aff51-533c-447a-8f19-219120730914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac64954f6e141d5b6e2cad4e2a0e535, user_id=729fdfabb5ad4592b04c14910a05ebeb, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.314045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aff51-533c-447a-8f19-219120730914 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.327611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1030/2086] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:32 2022] PUT /identity/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535/users/729fdfabb5ad4592b04c14910a05ebeb/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:32.330459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1057/2087] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:32.333375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aff51-533c-447a-8f19-219120730914 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.337976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.338721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.339151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.384133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] MFA Rules not processed for user `729fdfabb5ad4592b04c14910a05ebeb`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:32.385174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.412574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aff51-533c-447a-8f19-219120730914 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.420034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.420929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.421303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.421514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.422311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.424085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.438542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1031/2088] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:32.443066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.455243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b04aea-ed29-41d7-a95f-ed6ad8d6288d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.460655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.462025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.462452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.462774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.463492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.465595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.471656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1058/2089] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:32.477540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1032/2090] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:32.481539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.485025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.487047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.494544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98cc2-6ee9-410b-a7d2-39d30b32203c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.499226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.499958 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.500243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.500519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] PATH_INFO: `/v3/projects/db3681b9d7e443578d4c35689221846a/users/bf20c8f0c3e4442bba468d8b43243854/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.518417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3681b9d7e443578d4c35689221846a, user_id=bf20c8f0c3e4442bba468d8b43243854, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.521009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.535467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1059/2091] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:32 2022] PUT /identity/v3/projects/db3681b9d7e443578d4c35689221846a/users/bf20c8f0c3e4442bba468d8b43243854/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:32.544596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.544596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.563313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d585477-395f-4b88-9318-dfaa523c1d94 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.601751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6917-a7d1-4d72-9ed4-6e672964e7ac admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.607371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.608234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.608511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.608781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] PATH_INFO: `/v3/users/80ced7317f0f4652a51bad39252805b2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.610007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ced7317f0f4652a51bad39252805b2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.611973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.613858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.614560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.614842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.615118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.656752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ced7317f0f4652a51bad39252805b2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:32.657441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ced7317f0f4652a51bad39252805b2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:32.665629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ced7317f0f4652a51bad39252805b2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:32.666337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ced7317f0f4652a51bad39252805b2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:32.674183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ced7317f0f4652a51bad39252805b2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:32.677347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.678954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042765a3-2d83-48d9-b39e-c98385eecda0 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.689073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1033/2092] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:32.690349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.690670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.690967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.692990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1060/2093] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:32 2022] DELETE /identity/v3/users/80ced7317f0f4652a51bad39252805b2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:32.719563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] MFA Rules not processed for user `bf20c8f0c3e4442bba468d8b43243854`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:32.721124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.802725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1034/2094] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:32.807242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.809324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.891896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba90b07-f2af-4cbc-b70b-3172e260342a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.902457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.902457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.902457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.902457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.903137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.903137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.913936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.918481 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:32.931824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b900d-8244-4ed4-8989-f860461a9fd0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:32.937334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:32.938236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1061/2095] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:32.941369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:32.942432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:32.942432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] PATH_INFO: `/v3/projects/28efa99d77e4438ab681323ffb37d41e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:32.942640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:32.943703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28efa99d77e4438ab681323ffb37d41e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:32.946278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:32.999143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28efa99d77e4438ab681323ffb37d41e', 'request_id': 'req-96debee7-18d9-4a22-b431-5cbd607a9e12'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.009539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28efa99d77e4438ab681323ffb37d41e', 'request_id': 'req-96debee7-18d9-4a22-b431-5cbd607a9e12'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.018835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586fe75-4ad7-492c-9f07-10dcea4632f5 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.023198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28efa99d77e4438ab681323ffb37d41e', 'request_id': 'req-96debee7-18d9-4a22-b431-5cbd607a9e12'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.044303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] The token cache is being invalidate because project 28efa99d77e4438ab681323ffb37d41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:33.044604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.045341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96debee7-18d9-4a22-b431-5cbd607a9e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.047224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1035/2096] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:32 2022] DELETE /identity/v3/projects/28efa99d77e4438ab681323ffb37d41e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:33.051626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.051911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.052732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.073604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:33.075041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.085104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.086904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.087259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.087684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.088670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.090568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.120182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1062/2097] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:32 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:33.124430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.127022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.142439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064027b-8a40-4494-abae-852d09e326e2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.147444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.147949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1036/2098] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:33.148933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.150888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.151281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.151650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.152484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.154284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f269-c926-43c0-8358-56ec70ab3040 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.155421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.165964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1063/2099] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:33.168860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.169156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.169429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.171136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73533-17f0-4066-99c7-f4f0c9010c8f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.176046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.176931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.177215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.177508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] PATH_INFO: `/v3/users/4a86f0e32a0f4a43be6d2e955845d3c2` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.178758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a86f0e32a0f4a43be6d2e955845d3c2)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.181391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.188842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:33.190072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.205519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1064/2100] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:33.205519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.213463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d14261-f8d0-4aa4-9825-9d2e96834d3f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.218783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.219853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.220162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.220485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] PATH_INFO: `/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891/users/3c0fb2f407a74ecb927e05a02397fa56/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.237926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1770eb7fe534fdca5b6284d7d7c0891, user_id=3c0fb2f407a74ecb927e05a02397fa56, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.240169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a86f0e32a0f4a43be6d2e955845d3c2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.243367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.249007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a86f0e32a0f4a43be6d2e955845d3c2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.249867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a86f0e32a0f4a43be6d2e955845d3c2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.250732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a86f0e32a0f4a43be6d2e955845d3c2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.260661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a86f0e32a0f4a43be6d2e955845d3c2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:33.260830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1065/2101] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:33 2022] PUT /identity/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891/users/3c0fb2f407a74ecb927e05a02397fa56/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:33.263700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.263838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.289180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1037/2102] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:33 2022] DELETE /identity/v3/users/4a86f0e32a0f4a43be6d2e955845d3c2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:33.291624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.293046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.347241 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc1962-ae23-4454-8005-42d8140a9326 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.359712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a649e-58d4-42f0-b0d7-9a973e383f4c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.364439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.365113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.365397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.365674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.366420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.368560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c76bd-1625-46f9-8cf2-264f499593d4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.381405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1038/2103] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:33.381405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.381405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.381405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.398600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:33.399788 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.409180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1039/2104] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:33.409560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.409989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.410245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.410557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.410832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.411566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:33.411873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.413468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.419422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b81e3-099e-4221-aeab-56c96a023c2f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.424580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.428730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1066/2105] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:33.428897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.428897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.428897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] PATH_INFO: `/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891/users/3c0fb2f407a74ecb927e05a02397fa56/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.429129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.430719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1770eb7fe534fdca5b6284d7d7c0891, user_id=3c0fb2f407a74ecb927e05a02397fa56, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.431280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.433365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.443942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1040/2106] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:33 2022] PUT /identity/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891/users/3c0fb2f407a74ecb927e05a02397fa56/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:33.444990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.486535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8de61-8ee4-49db-8420-8cbfe2d92cdb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.505377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3035-b3de-4349-a0f2-305993295e94 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.514324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.515674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.516063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.516426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.519324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.520444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e3f2e-cc6a-491a-ab60-651d37b04234 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.540452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.541235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.541615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.541881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.551448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.552228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1041/2107] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:33.553897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.554186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.554501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.555849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.566830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1067/2108] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:33.567715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.582378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde40574-e75f-45b1-8490-102255a2099a service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.585214 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] MFA Rules not processed for user `3c0fb2f407a74ecb927e05a02397fa56`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:33.586474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.650177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.650885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.651508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.651508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.652117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.654036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.662254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1042/2109] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:33.674925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.679180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1068/2110] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:33.679361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.683731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.694273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edc60d-f9e4-4682-b7f6-0c6c175ee598 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.699696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.700410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.701148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.701432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.702176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.703813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.714378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1069/2111] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:33.718420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.730253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774d36-3a7b-4403-b91e-a9d0d09da29e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.735365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.736050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.736469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.736869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.737887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.739582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.752584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1070/2112] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:33.756646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.764960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424c1d9-91ab-49e7-a218-bcb98e875668 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.766092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fff8c-6b6a-4525-9f2a-fdc4de37057d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.768798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.769452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.769731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.770002 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.774045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.774980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.775255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.775531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] PATH_INFO: `/v3/projects/27ba9785cb054a34b8f075ed8a5d9c82/users/2bb3fc11d50c4862a1cd85a67235333e/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.784852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.786282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1c77-d859-4597-a5cb-f262ea998552 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.791521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ba9785cb054a34b8f075ed8a5d9c82, user_id=2bb3fc11d50c4862a1cd85a67235333e, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.794168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.796901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1043/2113] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:33.813524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1071/2114] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:33 2022] PUT /identity/v3/projects/27ba9785cb054a34b8f075ed8a5d9c82/users/2bb3fc11d50c4862a1cd85a67235333e/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:33.825283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.825652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.825933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.863135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.864916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.877443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] MFA Rules not processed for user `2bb3fc11d50c4862a1cd85a67235333e`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:33.879165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:33.944233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501359c-7396-47b7-b96e-1a96d20f0266 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.950845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.952190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.952341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.952620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] PATH_INFO: `/v3/projects/713227acbbb545acbbaf9fc35efd72a9` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:33.958225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=713227acbbb545acbbaf9fc35efd72a9)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:33.960175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:33.976791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1044/2115] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:33.978224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:33.992216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23546123-1ea9-4104-8a5b-8a29463a68cd None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:33.997486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:33.998369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:33.999182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:33.999464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] PATH_INFO: `/v3/users/bf20c8f0c3e4442bba468d8b43243854` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.000670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf20c8f0c3e4442bba468d8b43243854)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.002676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.007952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713227acbbb545acbbaf9fc35efd72a9', 'request_id': 'req-0a9c6f3e-7068-4ed2-9412-0153f634bf17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.013907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713227acbbb545acbbaf9fc35efd72a9', 'request_id': 'req-0a9c6f3e-7068-4ed2-9412-0153f634bf17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.019179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713227acbbb545acbbaf9fc35efd72a9', 'request_id': 'req-0a9c6f3e-7068-4ed2-9412-0153f634bf17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.032254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] The token cache is being invalidate because project 713227acbbb545acbbaf9fc35efd72a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:34.032670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.033401 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.038078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1072/2116] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:33 2022] DELETE /identity/v3/projects/713227acbbb545acbbaf9fc35efd72a9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:34.038078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.038078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.049018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf20c8f0c3e4442bba468d8b43243854'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.056785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf20c8f0c3e4442bba468d8b43243854'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.057523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf20c8f0c3e4442bba468d8b43243854'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.058169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf20c8f0c3e4442bba468d8b43243854'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.073570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf20c8f0c3e4442bba468d8b43243854'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.091107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1045/2117] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:33 2022] DELETE /identity/v3/users/bf20c8f0c3e4442bba468d8b43243854 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:34.091970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.093447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.115279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c6f3e-7068-4ed2-9412-0153f634bf17 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.158438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9062c-d7b9-4442-9dba-c670dd368629 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.163101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.163966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.164820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.165101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] PATH_INFO: `/v3/users/729fdfabb5ad4592b04c14910a05ebeb` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.166294 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=729fdfabb5ad4592b04c14910a05ebeb)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.168236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.179632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.180485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.180769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.181053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.183220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.215710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729fdfabb5ad4592b04c14910a05ebeb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.222887 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729fdfabb5ad4592b04c14910a05ebeb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.223662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729fdfabb5ad4592b04c14910a05ebeb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.224290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729fdfabb5ad4592b04c14910a05ebeb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.231873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-75a6b949-540d-45be-9879-a87fac045555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729fdfabb5ad4592b04c14910a05ebeb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.243035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.244645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.250574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1046/2118] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:34 2022] DELETE /identity/v3/users/729fdfabb5ad4592b04c14910a05ebeb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:34.251702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.251994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.252269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.257149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1073/2119] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:34.272704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:34.274643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.347534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1047/2120] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:34.363316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.363316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.464023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace9335-ed1e-4d02-9ed2-b20dbf459900 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.535976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.538800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.539615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.539615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.550969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.552450 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81424d69-4e33-473e-966e-8a4dcec05da4 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.569154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1074/2121] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:34.665235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.667800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.688066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987d05c-37c6-4fd9-a90c-7740e8a41554 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.694446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.696304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.696744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.697189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] PATH_INFO: `/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.699084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac64954f6e141d5b6e2cad4e2a0e535)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.702052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.761732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac64954f6e141d5b6e2cad4e2a0e535', 'request_id': 'req-431a893d-2950-4f3e-8919-9220116e1f90'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.767636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac64954f6e141d5b6e2cad4e2a0e535', 'request_id': 'req-431a893d-2950-4f3e-8919-9220116e1f90'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.774906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac64954f6e141d5b6e2cad4e2a0e535', 'request_id': 'req-431a893d-2950-4f3e-8919-9220116e1f90'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.789782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] The token cache is being invalidate because project 3ac64954f6e141d5b6e2cad4e2a0e535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:34.790271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.791399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-431a893d-2950-4f3e-8919-9220116e1f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.793497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1048/2122] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:34 2022] DELETE /identity/v3/projects/3ac64954f6e141d5b6e2cad4e2a0e535 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:34.798302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.798881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.799348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.819530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:34.821321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.881483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1075/2123] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:34.890803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.891095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.891377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.903818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.905293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.910371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:34.911584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:34.918314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7e2c2-b635-402c-b3a0-a9ebf4fa68e8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.923033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1049/2124] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:34.924245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.926287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.926287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.926287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] PATH_INFO: `/v3/projects/db3681b9d7e443578d4c35689221846a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.927305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3681b9d7e443578d4c35689221846a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.927706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.930773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.938914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566778ca-90f7-4d8c-a312-9701dfadc141 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.948458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:34.949122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:34.949402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:34.949675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:34.950414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:34.950760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:34.952383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:34.964479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1050/2125] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:34.968759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:34.978723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b63bf-2cd2-4450-bffd-7d5f74f278e6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:34.979468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3681b9d7e443578d4c35689221846a', 'request_id': 'req-de6016a1-b53f-42b8-952a-9a8d9189b4f4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.985575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3681b9d7e443578d4c35689221846a', 'request_id': 'req-de6016a1-b53f-42b8-952a-9a8d9189b4f4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:34.990756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3681b9d7e443578d4c35689221846a', 'request_id': 'req-de6016a1-b53f-42b8-952a-9a8d9189b4f4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.003893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] The token cache is being invalidate because project db3681b9d7e443578d4c35689221846a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:35.004181 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.004813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.006345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1076/2126] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:34 2022] DELETE /identity/v3/projects/db3681b9d7e443578d4c35689221846a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:35.010227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.011642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.037676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.038877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.040282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.040423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.041824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.045576 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.073795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1051/2127] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:34 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:35.077923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.079452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6016a1-b53f-42b8-952a-9a8d9189b4f4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.079577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.135926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.136747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.137024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.137304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] PATH_INFO: `/v3/users/3c0fb2f407a74ecb927e05a02397fa56` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.138527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0fb2f407a74ecb927e05a02397fa56)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.140304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517570a6-3d3a-43c3-9d23-eb0ae73f4b8b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.140465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.145128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.145789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.146071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.146340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.147040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.148923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.169567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1052/2128] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:35.173952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.183974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff369bd-0903-4deb-a281-dc3ca2012817 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.185033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0fb2f407a74ecb927e05a02397fa56'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.185692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0fb2f407a74ecb927e05a02397fa56'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.188615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.189252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.189554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.189831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.190565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.192139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.192498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0fb2f407a74ecb927e05a02397fa56'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.193143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0fb2f407a74ecb927e05a02397fa56'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.200655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0fb2f407a74ecb927e05a02397fa56'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.202729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1053/2129] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:35.206504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.208139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.218315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1077/2130] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:35 2022] DELETE /identity/v3/users/3c0fb2f407a74ecb927e05a02397fa56 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:35.220572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf8c16-4260-42b0-8fb5-1204df15a7a3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.280154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.280821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.281099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.281407 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.282236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.283942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cc513-72ed-42d8-826c-cf2eb9c6bdca admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.298372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1054/2131] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:35.299529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.309979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1255f0-f331-4ad1-b727-b632cf7de23c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.314648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.315498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.315776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.316049 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] PATH_INFO: `/v3/projects/e0ebc6098c5045dea82905fe88480a69/users/b3a43771cc4c43aebd1d3f8c98bc6913/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.332294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ebc6098c5045dea82905fe88480a69, user_id=b3a43771cc4c43aebd1d3f8c98bc6913, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.335028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.354435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1078/2132] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:35 2022] PUT /identity/v3/projects/e0ebc6098c5045dea82905fe88480a69/users/b3a43771cc4c43aebd1d3f8c98bc6913/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:35.362038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.362330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.362639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.391523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] MFA Rules not processed for user `b3a43771cc4c43aebd1d3f8c98bc6913`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:35.392651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.468652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1055/2133] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:35.479823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.481553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.504800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.506249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.552243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43299-620e-4566-b5f9-d5864a7d4834 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.557439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.558096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.558613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.558805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.567728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.569139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.578335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1079/2134] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:35.579267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0adac-5994-440a-9a27-4cf8ed7d1e3c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.584014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.585017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.585415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.585873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] PATH_INFO: `/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.587118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1770eb7fe534fdca5b6284d7d7c0891)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.589102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.599869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.612232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac57866-758a-4a96-9dbc-505b9e43e178 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.616913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.618302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.618613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.618927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] PATH_INFO: `/v3/users/b3a43771cc4c43aebd1d3f8c98bc6913` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.620251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a43771cc4c43aebd1d3f8c98bc6913)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.622546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.646768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1770eb7fe534fdca5b6284d7d7c0891', 'request_id': 'req-bc3546a7-9bf6-418d-a553-e274dbf3ad87'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.652618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1770eb7fe534fdca5b6284d7d7c0891', 'request_id': 'req-bc3546a7-9bf6-418d-a553-e274dbf3ad87'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.659917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1770eb7fe534fdca5b6284d7d7c0891', 'request_id': 'req-bc3546a7-9bf6-418d-a553-e274dbf3ad87'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.673792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] The token cache is being invalidate because project a1770eb7fe534fdca5b6284d7d7c0891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:35.674080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.674750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc3546a7-9bf6-418d-a553-e274dbf3ad87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.676525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1056/2135] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:35 2022] DELETE /identity/v3/projects/a1770eb7fe534fdca5b6284d7d7c0891 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:35.676635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a43771cc4c43aebd1d3f8c98bc6913'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.677190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a43771cc4c43aebd1d3f8c98bc6913'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.684246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1e47-61fd-4008-b398-37118285b924 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.684246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1e47-61fd-4008-b398-37118285b924 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.684246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1e47-61fd-4008-b398-37118285b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.688660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a43771cc4c43aebd1d3f8c98bc6913'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.689380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a43771cc4c43aebd1d3f8c98bc6913'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.701825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7ffaa9c1-f33d-473a-a356-d2b5b16e6b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a43771cc4c43aebd1d3f8c98bc6913'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:35.703179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-108e1e47-61fd-4008-b398-37118285b924 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:35.704517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-108e1e47-61fd-4008-b398-37118285b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.725346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1080/2136] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:35 2022] DELETE /identity/v3/users/b3a43771cc4c43aebd1d3f8c98bc6913 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:35.730290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.730701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.730978 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.748614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:35.749845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.772890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1057/2137] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:35.782416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.782727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.782993 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.801251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:35.802476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.812912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1081/2138] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:35.822076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.824001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:35.865385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1058/2139] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:35.870244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.879634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed7af5-ab26-4228-8b1c-dc1ac843dfe5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.884326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.884827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.885101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.885373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.886088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:35.886451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.888035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.891955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f59cc-9eea-4d2f-83f8-438a8141a316 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.897054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.897936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.898300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.898736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.904166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1059/2140] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:35.905169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.911280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.913078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.917136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408a74-cd09-45a4-b6d4-35c5fb0e2f3b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:35.927511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:35.928281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:35.928636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:35.929003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:35.930337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:35.932417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:35.934301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1082/2141] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:35.968267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1060/2142] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:35.972433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:35.984680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747e39-32ac-4e7f-985d-34d2e53e047c service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.055696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.056383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.056665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.056952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.057619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.063550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.104770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1083/2143] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:35 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:36.105959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.122926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b4c32-42d0-4e09-8a94-28a7320749a1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.129005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.129785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.130136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.130526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.131305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.140064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.154434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1061/2144] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:36.162605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.172124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dc62b-c247-4c0a-b300-a6701d7eabcd admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.185004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.185781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.186064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.186335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.187272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.189237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.205090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1084/2145] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:36.205538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.215901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3767-a5ac-40e5-b5cb-66d9d4328ff9 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.220571 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.221471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.221750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.222022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] PATH_INFO: `/v3/projects/2954db13c70c43b98c05d392089b3a93/users/327ee22061174279a140e12445039b60/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.236773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2954db13c70c43b98c05d392089b3a93, user_id=327ee22061174279a140e12445039b60, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.239723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee95d5-cefa-4d7d-845d-cac4dc79aae2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.250128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.251961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.254442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1062/2146] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:36 2022] PUT /identity/v3/projects/2954db13c70c43b98c05d392089b3a93/users/327ee22061174279a140e12445039b60/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:36.265310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b256-c291-4551-8721-8b418ff8bd49 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.265480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221be-12f5-4213-b347-fde80a3d494d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.265809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221be-12f5-4213-b347-fde80a3d494d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.266019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221be-12f5-4213-b347-fde80a3d494d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.297954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-751221be-12f5-4213-b347-fde80a3d494d None None] MFA Rules not processed for user `327ee22061174279a140e12445039b60`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:36.297954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-751221be-12f5-4213-b347-fde80a3d494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.332933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.333805 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.334088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.334369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] PATH_INFO: `/v3/projects/e0ebc6098c5045dea82905fe88480a69` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.335614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ebc6098c5045dea82905fe88480a69)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.338062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.360419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1063/2147] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:36.372496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751221be-12f5-4213-b347-fde80a3d494d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.373921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-751221be-12f5-4213-b347-fde80a3d494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.387441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ebc6098c5045dea82905fe88480a69', 'request_id': 'req-ed6eaead-b476-4177-a8bb-551947a7f3a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:36.393376 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ebc6098c5045dea82905fe88480a69', 'request_id': 'req-ed6eaead-b476-4177-a8bb-551947a7f3a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:36.399010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ebc6098c5045dea82905fe88480a69', 'request_id': 'req-ed6eaead-b476-4177-a8bb-551947a7f3a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:36.413702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] The token cache is being invalidate because project e0ebc6098c5045dea82905fe88480a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:36.413889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:36.415234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-ed6eaead-b476-4177-a8bb-551947a7f3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:36.416812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1085/2148] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:36 2022] DELETE /identity/v3/projects/e0ebc6098c5045dea82905fe88480a69 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:36.421649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9fe2-bc77-4bf1-9927-62632b81a868 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.421967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9fe2-bc77-4bf1-9927-62632b81a868 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.422267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9fe2-bc77-4bf1-9927-62632b81a868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.432024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751221be-12f5-4213-b347-fde80a3d494d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.440051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-826c9fe2-bc77-4bf1-9927-62632b81a868 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:36.441250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-826c9fe2-bc77-4bf1-9927-62632b81a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.497449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.498108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.498404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.498693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.500525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.501778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1086/2149] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:36.511969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.512257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.512551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.532108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:36.533295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:36.544638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1087/2150] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:36.548702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.558847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134ff2-1bbe-43ff-bc76-4053810876a5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.563495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.564132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.564412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.564686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.565404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:36.565708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.566815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.567327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.568245 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.579732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1064/2151] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:36.579935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1088/2152] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:04:36.583676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.595357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454823f-da5e-4200-b92f-806eae12d467 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.600139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.600791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.601073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.601342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.602218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.604448 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.633369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1065/2153] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:36.634596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.644418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403014f-23a5-4500-b73e-986e1544cee9 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.703506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.704146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.704428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.704698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.705438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.707462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.731412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1089/2154] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:36.732826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.744207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478afa90-729f-4638-a79d-7223f70f88e8 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.749677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.750383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.750715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.750986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.751706 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.753290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.768232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1066/2155] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:36.769414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.780504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5a87-2374-489e-a1e1-cc544bd4a103 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.787237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.787849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.788172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.788481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.789295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.790989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c44b-1b18-4b48-ac9c-fec9e7afe561 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.805133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1090/2156] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:36.809350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:36.821054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255086-53b4-4eea-8921-37e7537ac487 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:36.842953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:36.844689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.845109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.845552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] PATH_INFO: `/v3/projects/bc972118412b4aba9a1fdcd9be071203/users/8a8acee14a9e41e5b02bd59ec8c3828e/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.870707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc972118412b4aba9a1fdcd9be071203, user_id=8a8acee14a9e41e5b02bd59ec8c3828e, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:36.873636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:36.895173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1067/2157] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:36 2022] PUT /identity/v3/projects/bc972118412b4aba9a1fdcd9be071203/users/8a8acee14a9e41e5b02bd59ec8c3828e/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:36.906074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b49d-d9c0-4e8f-90c9-0e3e2d911a80 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:36.906542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b49d-d9c0-4e8f-90c9-0e3e2d911a80 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:36.906716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b49d-d9c0-4e8f-90c9-0e3e2d911a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:36.943264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-bb00b49d-d9c0-4e8f-90c9-0e3e2d911a80 None None] MFA Rules not processed for user `8a8acee14a9e41e5b02bd59ec8c3828e`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:36.944503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bb00b49d-d9c0-4e8f-90c9-0e3e2d911a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:37.008685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1091/2158] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:37.019890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:37.021556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:37.081101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff32b-2ea2-41ab-a580-5eed5ea8f538 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:37.084820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:37.085444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:37.085730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:37.086006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:37.094093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:37.095546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:37.103961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1068/2159] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:41.083289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82a62-d263-4edf-b0e4-f96dea87a901 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:41.083809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82a62-d263-4edf-b0e4-f96dea87a901 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:41.083925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a82a62-d263-4edf-b0e4-f96dea87a901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:41.101298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-05a82a62-d263-4edf-b0e4-f96dea87a901 None None] MFA Rules not processed for user `327ee22061174279a140e12445039b60`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:41.103154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-05a82a62-d263-4edf-b0e4-f96dea87a901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:41.160615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1092/2160] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:43.208995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:43.270987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320185c6-1ed1-43e7-974f-d403c41dff9c service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:43.276154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:43.277008 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:43.277287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:43.277558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] PATH_INFO: `/v3/users/8a8acee14a9e41e5b02bd59ec8c3828e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:43.278752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8acee14a9e41e5b02bd59ec8c3828e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:43.280664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:43.324344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8acee14a9e41e5b02bd59ec8c3828e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.332806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8acee14a9e41e5b02bd59ec8c3828e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.333519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8acee14a9e41e5b02bd59ec8c3828e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.334120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8acee14a9e41e5b02bd59ec8c3828e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.341668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8acee14a9e41e5b02bd59ec8c3828e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.357870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1069/2161] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:43 2022] DELETE /identity/v3/users/8a8acee14a9e41e5b02bd59ec8c3828e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:43.362567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:43.363124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:43.363124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:43.382443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:43.384448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:43.444654 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1093/2162] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:43.454848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:43.456217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:43.512629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5408c-6a6b-41af-aee1-65c80307e95d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:43.516277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:43.516924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:43.517207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:43.517482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:43.526832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:43.528256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeef4b-b12a-45fa-ac82-9dce86a31927 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:43.537541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1070/2163] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:43.796745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:43.798483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:43.812065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864264d-35f5-4611-9086-ae91aa99e63b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:43.818553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:43.819432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:43.819724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:43.820001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] PATH_INFO: `/v3/projects/bc972118412b4aba9a1fdcd9be071203` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:43.822130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc972118412b4aba9a1fdcd9be071203)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:43.830345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:43.880314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc972118412b4aba9a1fdcd9be071203', 'request_id': 'req-e53112dd-c1e1-4a4a-9f7e-4916679f07de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.888803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc972118412b4aba9a1fdcd9be071203', 'request_id': 'req-e53112dd-c1e1-4a4a-9f7e-4916679f07de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.894140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc972118412b4aba9a1fdcd9be071203', 'request_id': 'req-e53112dd-c1e1-4a4a-9f7e-4916679f07de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.906599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] The token cache is being invalidate because project bc972118412b4aba9a1fdcd9be071203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:43.907524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.909731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e53112dd-c1e1-4a4a-9f7e-4916679f07de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:43.911510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1094/2164] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:43 2022] DELETE /identity/v3/projects/bc972118412b4aba9a1fdcd9be071203 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:43.916476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:43.916766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:43.917151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:43.939667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:43.940888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.004138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1071/2165] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:44.007224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.007548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.007858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.032775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:44.032775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.043247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1095/2166] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:44.047290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.057896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9ad3-14ae-41fa-8422-c5195199d94b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.063251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.063986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.064397 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.064761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.065740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:44.066219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.068153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.081612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1072/2167] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:44.085708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.095393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef30f68-f486-4df2-b207-9c81c0b061a4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.100396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.101056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.101943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.102237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.102972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.105028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.126243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.127972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.137552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1096/2168] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:44 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:44.140698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eeb62-8a19-4565-bf8d-43d60ded4d6b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.141489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.195367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.196423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.196779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.197058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] PATH_INFO: `/v3/users/2bb3fc11d50c4862a1cd85a67235333e` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.198347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb3fc11d50c4862a1cd85a67235333e)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.200819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.203837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57831d-a69a-4f52-9665-6dba4c5624af admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.208837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.209544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.209826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.210108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.211267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.216823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.244321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1097/2169] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:44 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:44.246807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.256826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb3fc11d50c4862a1cd85a67235333e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:44.257349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df468593-2325-45c4-bb87-3c01ee4d0a27 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.262370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.263065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.263346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.263622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.264373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.264596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb3fc11d50c4862a1cd85a67235333e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:44.265195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb3fc11d50c4862a1cd85a67235333e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:44.265819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb3fc11d50c4862a1cd85a67235333e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:44.265990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.275905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-069a7118-2cdb-4215-a9a9-153afd270f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb3fc11d50c4862a1cd85a67235333e'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:44.280656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1098/2170] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:44.281694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.283464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.293058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1073/2171] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:44 2022] DELETE /identity/v3/users/2bb3fc11d50c4862a1cd85a67235333e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:44.296419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb0937-1ac1-4ba0-b6d7-165c4e06c31e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.297562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.297847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.298123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.314960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:44.316128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.349651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.350316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.350624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.350896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.351634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.353242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.364044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1099/2172] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:44.368641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.376827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1074/2173] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:44.378708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926ea0e-7279-4aa7-9674-339f2b353ffb admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.383871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.384739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.385021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.385295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] PATH_INFO: `/v3/projects/c9d3d605483a4228bd54793a27723c91/users/3bd41b34f5b14ae0a0ebccec9d59944f/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.387262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.388682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.400803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d3d605483a4228bd54793a27723c91, user_id=3bd41b34f5b14ae0a0ebccec9d59944f, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.403496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a5065-bfcd-4895-a0e3-150edae63558 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.416951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1100/2174] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:44 2022] PUT /identity/v3/projects/c9d3d605483a4228bd54793a27723c91/users/3bd41b34f5b14ae0a0ebccec9d59944f/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:44.421999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.422287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.422594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.441320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:44.442641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.445904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191da78-06c5-4ed2-bc01-3529b272eb32 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.491050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.491696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.492486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.492777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.499519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1101/2175] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:44.502198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.503760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.511188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.515240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1075/2176] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:44.520710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b28c3c-bc53-4cb2-9676-9f67cbd585eb None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.524402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.525042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.525326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.525600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.534578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.535993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922ee02-3bac-4694-a508-982532c2a537 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.545199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1102/2177] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:44.891926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:44.891926 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:44.905717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cf0af-3bcb-4e89-aa0f-4537cf921184 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:44.912498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:44.914987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:44.914987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:44.916194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] PATH_INFO: `/v3/projects/27ba9785cb054a34b8f075ed8a5d9c82` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:44.917558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ba9785cb054a34b8f075ed8a5d9c82)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:44.919967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:44.995188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ba9785cb054a34b8f075ed8a5d9c82', 'request_id': 'req-b82a7f26-b932-451a-ac68-4bef07c210e1'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:45.001090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ba9785cb054a34b8f075ed8a5d9c82', 'request_id': 'req-b82a7f26-b932-451a-ac68-4bef07c210e1'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:45.008511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ba9785cb054a34b8f075ed8a5d9c82', 'request_id': 'req-b82a7f26-b932-451a-ac68-4bef07c210e1'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:45.026774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] The token cache is being invalidate because project 27ba9785cb054a34b8f075ed8a5d9c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:45.027066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:45.027807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-b82a7f26-b932-451a-ac68-4bef07c210e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:45.033470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1076/2178] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:44 2022] DELETE /identity/v3/projects/27ba9785cb054a34b8f075ed8a5d9c82 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:45.034922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.035519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.035989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.068123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:45.069828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:45.154470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1103/2179] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:45.162300 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.162710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.163029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.185164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:45.186381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:45.194263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1077/2180] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:45.198618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.207985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4bd8b-ec04-4518-b65b-086cf2f85c2d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.213146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.213800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.214077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.214349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.215097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:45.215399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.217457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.241295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1104/2181] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:45.245613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.255146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8b33e-5aa7-488e-bb1c-ef62456b5a92 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.259955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.260586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.260679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.260933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.261594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.263653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.292845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1078/2182] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:45.297268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.307565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358d58-ab96-489b-a3cc-8e3945dcaf09 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.367199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.367867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.368145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.368416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.369099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.371076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a459be-0e53-4be8-a797-0ccfea9a9695 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.391580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1105/2183] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:45.395688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.409823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999007-be0e-4335-b8f5-0d98b6149e03 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.417514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.418588 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.419368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.419853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.420795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.422673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.423604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.423894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.424189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.436843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1079/2184] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:45.438565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.448746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1474-b597-4c74-ac16-47e8eb9731be admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.452411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] MFA Rules not processed for user `3bd41b34f5b14ae0a0ebccec9d59944f`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:45.453633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:45.454320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.454638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.454990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.455344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.456511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.457857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.472484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1080/2185] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:45.473573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.484878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0059d-5988-4bcb-ae18-58e230b99b99 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.489729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.490710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.490922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.491237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] PATH_INFO: `/v3/projects/0d7e87d27bea40e6b4d9da10d0e525c9/users/544caa66833a4c7887616a3800e8bb1f/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.510103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7e87d27bea40e6b4d9da10d0e525c9, user_id=544caa66833a4c7887616a3800e8bb1f, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.513203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c56b3-a84c-4461-b6ef-d86caac4eab1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.529022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1081/2186] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:45 2022] PUT /identity/v3/projects/0d7e87d27bea40e6b4d9da10d0e525c9/users/544caa66833a4c7887616a3800e8bb1f/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:45.532747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1106/2187] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:04:45.543199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.543488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.543848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.544077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.545490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:45.580995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] MFA Rules not processed for user `544caa66833a4c7887616a3800e8bb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:45.582509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:45.610955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eff26e-ac81-4703-a7fc-8875f0bd5f17 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.646213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1082/2188] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:45.651513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.665072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.665863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.666152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.666449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.712042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa44395-95cb-4d15-ab3b-58164b2442cf None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.716824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.717472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.717752 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.718027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.718747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.720629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.720741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.722115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.732275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1107/2189] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:45.754804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1083/2190] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:45.755959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.766565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4daa27-f9a2-4905-af7a-03b2e7480ff4 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.822849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.823521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.823804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.824082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.824769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.826696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.848191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1108/2191] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:45 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:45.849458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.860432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea0a6-73b0-4161-8492-8b3d2176f1fe admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.865659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.866370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.866709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.867012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.868275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.870180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.882509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1084/2192] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:04:45.887143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.899966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dc965-838d-4b22-9fa8-afed364c9d93 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:45.912597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:45.914614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:45.915833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:45.916185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] PATH_INFO: `/v3/projects/21b1ff61b7944391be7fac55d092b194/users/246efb9a38a544f292aa19208ef33290/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:45.939970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b1ff61b7944391be7fac55d092b194, user_id=246efb9a38a544f292aa19208ef33290, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:45.942707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:45.955899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1109/2193] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:45 2022] PUT /identity/v3/projects/21b1ff61b7944391be7fac55d092b194/users/246efb9a38a544f292aa19208ef33290/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:45.960084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:45.975745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc7b84-56c8-4490-b3d7-0dc84e33766e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:46.046482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:46.047400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.047747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.047963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.048685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:46.050304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382fe4-b359-4cf5-b93f-13d3cb8e76e5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:46.063442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1085/2194] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:46.065003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:46.075844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fdafe-9477-47e7-83a3-4102de8c91f7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:46.080470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:46.081346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.081622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.081949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] PATH_INFO: `/v3/projects/21b1ff61b7944391be7fac55d092b194/users/246efb9a38a544f292aa19208ef33290/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.084093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b1ff61b7944391be7fac55d092b194, user_id=246efb9a38a544f292aa19208ef33290, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:46.087419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:46.097338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1110/2195] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:46 2022] PUT /identity/v3/projects/21b1ff61b7944391be7fac55d092b194/users/246efb9a38a544f292aa19208ef33290/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:46.109623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.109911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.110190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.135808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] MFA Rules not processed for user `246efb9a38a544f292aa19208ef33290`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:46.137006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:46.202647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1086/2196] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:46.214075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:46.215878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:46.271948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d537-0a52-4cae-8f50-56a7870ad664 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:46.276182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:46.276812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.277092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.277364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.286008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:46.287443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a3f7-bb0c-4ae5-b416-391c29680d03 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:46.296765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1111/2197] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:46.698645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:46.762132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a70223-af3e-4653-93ae-da9a1f3a382f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:46.766634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:46.767530 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.767807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.768078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] PATH_INFO: `/v3/users/544caa66833a4c7887616a3800e8bb1f` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.769298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544caa66833a4c7887616a3800e8bb1f)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:46.771435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:46.815795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544caa66833a4c7887616a3800e8bb1f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:46.822946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544caa66833a4c7887616a3800e8bb1f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:46.823618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544caa66833a4c7887616a3800e8bb1f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:46.824219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544caa66833a4c7887616a3800e8bb1f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:46.830992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544caa66833a4c7887616a3800e8bb1f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:46.848060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1087/2198] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:46 2022] DELETE /identity/v3/users/544caa66833a4c7887616a3800e8bb1f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:46.852559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa625-4dc3-4871-8969-ba0437264ccb None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:46.853041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa625-4dc3-4871-8969-ba0437264ccb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:46.853152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa625-4dc3-4871-8969-ba0437264ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:46.871669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8bcaa625-4dc3-4871-8969-ba0437264ccb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:46.872884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8bcaa625-4dc3-4871-8969-ba0437264ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:46.941080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1112/2199] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:46.951736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:46.953201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:47.013858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960d682-f927-4b07-81cf-92db4464d713 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:47.018248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:47.018517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:47.018805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:47.019077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:47.028987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:47.030503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:47.041221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1088/2200] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:04:47.102101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:47.102402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:47.102693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:47.122826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] MFA Rules not processed for user `3bd41b34f5b14ae0a0ebccec9d59944f`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:47.124810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:47.187708 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1113/2201] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:47.335447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:47.337364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:47.353332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133af46e-6089-4b1f-ab01-16c0a076f3f2 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:47.358210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:47.359125 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:47.359514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:47.361640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] PATH_INFO: `/v3/projects/0d7e87d27bea40e6b4d9da10d0e525c9` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:47.362920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7e87d27bea40e6b4d9da10d0e525c9)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:47.364822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:47.416077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7e87d27bea40e6b4d9da10d0e525c9', 'request_id': 'req-24f9576c-4fef-4145-aaea-6fd27b8eecea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.421672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7e87d27bea40e6b4d9da10d0e525c9', 'request_id': 'req-24f9576c-4fef-4145-aaea-6fd27b8eecea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.427814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7e87d27bea40e6b4d9da10d0e525c9', 'request_id': 'req-24f9576c-4fef-4145-aaea-6fd27b8eecea'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.440452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] The token cache is being invalidate because project 0d7e87d27bea40e6b4d9da10d0e525c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:47.440750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.441528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.444291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1089/2202] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:47 2022] DELETE /identity/v3/projects/0d7e87d27bea40e6b4d9da10d0e525c9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:47.447696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:47.458732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa95b6-ac3b-4133-9f72-859fe8c478b9 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:47.517605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:47.518508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:47.518794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:47.519074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] PATH_INFO: `/v3/users/246efb9a38a544f292aa19208ef33290` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:47.520255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246efb9a38a544f292aa19208ef33290)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:47.522205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:47.570235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246efb9a38a544f292aa19208ef33290'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.571329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246efb9a38a544f292aa19208ef33290'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.580062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246efb9a38a544f292aa19208ef33290'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.580721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246efb9a38a544f292aa19208ef33290'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.591979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-1b059437-54dd-4c25-b865-fdf8535d2947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246efb9a38a544f292aa19208ef33290'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:47.612554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1114/2203] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:04:47 2022] DELETE /identity/v3/users/246efb9a38a544f292aa19208ef33290 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:47.878309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:47.879824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:47.893563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9576c-4fef-4145-aaea-6fd27b8eecea admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:47.953860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:47.954730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:47.955009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:47.955306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] PATH_INFO: `/v3/projects/21b1ff61b7944391be7fac55d092b194` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:47.956704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b1ff61b7944391be7fac55d092b194)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:47.959042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.005693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b1ff61b7944391be7fac55d092b194', 'request_id': 'req-de498cff-dd94-4274-88bd-b6e39440cf61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:48.011500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b1ff61b7944391be7fac55d092b194', 'request_id': 'req-de498cff-dd94-4274-88bd-b6e39440cf61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:48.017629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b1ff61b7944391be7fac55d092b194', 'request_id': 'req-de498cff-dd94-4274-88bd-b6e39440cf61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:48.031207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] The token cache is being invalidate because project 21b1ff61b7944391be7fac55d092b194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:04:48.031492 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:48.032118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-de498cff-dd94-4274-88bd-b6e39440cf61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:04:48.033779 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1090/2204] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:04:47 2022] DELETE /identity/v3/projects/21b1ff61b7944391be7fac55d092b194 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:48.039633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.040179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.040339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.058801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:48.060002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:48.119210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1115/2205] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:48.127279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.127465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.127908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.147227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:48.148425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:48.156187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1091/2206] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:48.160418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.169879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037b9cd-b56d-4fb4-b494-3980d8d23b29 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.174657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.175310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.175595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.176108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.176980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:04:48.177291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.179163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.191218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1116/2207] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:04:48.196187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.205434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a355bf-eaff-4df8-9613-6c96121bd978 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.210414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.211310 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.211593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.211865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.212569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.214508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.239179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1092/2208] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:48.243870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.253720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc641214-1ad7-4c1c-8ea4-5f45a1fdd682 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.308897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.309575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.309861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.310766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.311517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.313463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.336315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1117/2209] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/users => generated 278 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:04:48.337774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.348290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d30b6-1771-43fb-95ff-9f39090ce3e0 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.353692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.354347 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.354652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.354921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.355628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.357285 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.367869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1093/2210] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:48.371980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.381828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec26b2-c14a-456a-a219-df1a11ead42f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.386564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.387228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.387505 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.387776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.388493 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.390097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e24c1-a9e2-4cc8-ad1f-9d3ca40e01b2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.400619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1118/2211] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:48.404489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.414878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5b765-577d-4dc7-b3b2-1f7e0a3739dc admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.419354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.420208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.420486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.420758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] PATH_INFO: `/v3/projects/521cd935894f4fd8aba2c88e1fdec534/users/4a6203c822d64dd09651a3683b7fba8d/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.435749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=521cd935894f4fd8aba2c88e1fdec534, user_id=4a6203c822d64dd09651a3683b7fba8d, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.438409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.451544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1094/2212] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:48 2022] PUT /identity/v3/projects/521cd935894f4fd8aba2c88e1fdec534/users/4a6203c822d64dd09651a3683b7fba8d/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:48.462359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.462665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.462937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.488728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] MFA Rules not processed for user `4a6203c822d64dd09651a3683b7fba8d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:48.489946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:48.553542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1119/2213] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:48.558291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.618917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc64-60c1-4e40-874f-ca247f6355ef admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.623787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.624443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.624726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.625001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.625699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.627638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.654959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1095/2214] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:48.656869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.667565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ae488-08ac-451a-af78-da82a64c3284 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.722860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.723588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.723873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.724159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.724864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.726879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.747946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1120/2215] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:04:48.751925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.762574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2e365-671e-4160-b497-d44b24883909 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.767058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.767698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.767979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.768251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.768953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.770561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.781151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1096/2216] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:48.784934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.794946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc763cdb-4587-44bc-8ef7-57187d188a06 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.799765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.800638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.800917 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.801190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] PATH_INFO: `/v3/projects/b90053f5060a4ba1a6b9434f675960c0/users/723342a7b9ca4a05b528d77d03e83fe2/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.817581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90053f5060a4ba1a6b9434f675960c0, user_id=723342a7b9ca4a05b528d77d03e83fe2, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.820892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.833605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1121/2217] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:48 2022] PUT /identity/v3/projects/b90053f5060a4ba1a6b9434f675960c0/users/723342a7b9ca4a05b528d77d03e83fe2/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:04:48.838123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.847892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40f6a-19d1-4ffd-b875-112f782a90f6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.902693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.903417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.903648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.903958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.904682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.906410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a0845-f326-4809-8925-d96789404daf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.916122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1097/2218] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:04:48.920504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:48.931339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489960-b186-4275-83b3-d0e4749f3ca3 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:48.935856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:48.936729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.937007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.937277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] PATH_INFO: `/v3/projects/b90053f5060a4ba1a6b9434f675960c0/users/723342a7b9ca4a05b528d77d03e83fe2/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.939539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90053f5060a4ba1a6b9434f675960c0, user_id=723342a7b9ca4a05b528d77d03e83fe2, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:48.942143 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:48.953020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1122/2219] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:04:48 2022] PUT /identity/v3/projects/b90053f5060a4ba1a6b9434f675960c0/users/723342a7b9ca4a05b528d77d03e83fe2/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:04:48.964509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:48.964855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:48.965107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:48.993202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] MFA Rules not processed for user `723342a7b9ca4a05b528d77d03e83fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:04:48.994375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:49.061596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1098/2220] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:04:49.074885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:49.076662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:04:49.133568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e06055-9aba-428f-b47f-981cf5fb6abc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:49.138575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:49.140392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:49.140392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:49.140392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:49.150194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:49.151670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:49.161340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1123/2221] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:04:49.238693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:04:49.249592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56bc7-e008-4a25-9fa7-00314b31d936 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:04:49.253461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:04:49.254109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:04:49.254410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:04:49.254697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:04:49.309301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:04:49.310778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6fc60-3462-4993-b86d-a44c1dcb0c2e service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:04:49.319480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1099/2222] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:08.419938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:08.421752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:08.570632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076679f-d1ad-45d3-93fa-f2b87a2a3e22 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:08.575481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:08.576630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:08.576916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:08.577194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] PATH_INFO: `/v3/users/c0cd307eb71146e5a99f3ced3664b9ed` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:08.578468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0cd307eb71146e5a99f3ced3664b9ed)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:08.581051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:08.634576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0cd307eb71146e5a99f3ced3664b9ed'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:08.635243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0cd307eb71146e5a99f3ced3664b9ed'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:08.642050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0cd307eb71146e5a99f3ced3664b9ed'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:08.643147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0cd307eb71146e5a99f3ced3664b9ed'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:08.651447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0cd307eb71146e5a99f3ced3664b9ed'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:08.670407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1124/2223] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:08 2022] DELETE /identity/v3/users/c0cd307eb71146e5a99f3ced3664b9ed => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:08.675386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:08.675940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:08.676045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:08.709747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:08.711616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:08.776037 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1100/2224] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:08.785779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:08.787307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:08.882923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d6532-e7d6-4034-8e54-d1ad33eb61de admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:08.886546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:08.887190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:08.887475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:08.887754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:08.896412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:08.898487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232d9a0-0e27-427a-ba98-e9f1ba1469d0 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:08.907710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1125/2225] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:09.159811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.161232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.173493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f166-36cb-4da0-a00a-efef429c149d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.178312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.179739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.180044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.180435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] PATH_INFO: `/v3/projects/f9fb651c72564bd0a8fe5090978fc50d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.181689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9fb651c72564bd0a8fe5090978fc50d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.183838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.234953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9fb651c72564bd0a8fe5090978fc50d', 'request_id': 'req-6e851cdf-7137-4aa7-94f8-91534cd193d9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.240183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9fb651c72564bd0a8fe5090978fc50d', 'request_id': 'req-6e851cdf-7137-4aa7-94f8-91534cd193d9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.246986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9fb651c72564bd0a8fe5090978fc50d', 'request_id': 'req-6e851cdf-7137-4aa7-94f8-91534cd193d9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.260104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] The token cache is being invalidate because project f9fb651c72564bd0a8fe5090978fc50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:09.260388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.261007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e851cdf-7137-4aa7-94f8-91534cd193d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.262725 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1101/2226] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:09 2022] DELETE /identity/v3/projects/f9fb651c72564bd0a8fe5090978fc50d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:09.267921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.268239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.268519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.315479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:09.316692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.373715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1126/2227] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:09.381375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.381664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.381943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.401047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:09.402357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.414477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1102/2228] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:09.415440 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.426182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134880e-676c-4a3b-8449-05aafc49a6f3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.434029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.434029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.435769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.435769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.436117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:09.436633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.439092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.455503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1127/2229] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:09.456733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.466947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b1aa-de0c-4e69-abf3-631387eaafe4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.472349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.472923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.473198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.473466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.474150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.476661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.511743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1103/2230] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:09.515639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.528422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd1db-b763-4a19-82ae-67fcc642f5f0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.605851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.606588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.606871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.607160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.608093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.611424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.660393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1128/2231] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:09.666231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.678479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308a05e-8c7f-49bc-9fd7-e2e61d8ed4f7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.702466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.703536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.703890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.704264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.705101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.706930 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.707468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.709227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.718950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1104/2232] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:09.722482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.723759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da842-ad9b-4cbb-85f0-0b3fb3b5eb29 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.730893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.732670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.733191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.733578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] PATH_INFO: `/v3/users/4a6203c822d64dd09651a3683b7fba8d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.734269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c93-e67e-47ba-99c7-d7ee75508646 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.735003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6203c822d64dd09651a3683b7fba8d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.739504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.740488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.741348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.742365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.742848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.743575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.745166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.761717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1105/2233] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:05:09.766497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.782011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c32032-d7ac-4a6b-b249-7e1173caec71 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:09.788934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:09.790079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.790363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.790683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] PATH_INFO: `/v3/projects/2ba79aa5a25b4cb98a2c5ce25e1b783d/users/4b895c0c03ae4b398ca24924f3f27dd3/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.818924 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba79aa5a25b4cb98a2c5ce25e1b783d, user_id=4b895c0c03ae4b398ca24924f3f27dd3, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:09.821968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75ac88-b0ef-4aff-8370-9e556ca6fd0e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:09.831301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6203c822d64dd09651a3683b7fba8d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.832142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6203c822d64dd09651a3683b7fba8d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.849159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1106/2234] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:09 2022] PUT /identity/v3/projects/2ba79aa5a25b4cb98a2c5ce25e1b783d/users/4b895c0c03ae4b398ca24924f3f27dd3/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:09.849569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6203c822d64dd09651a3683b7fba8d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.849569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6203c822d64dd09651a3683b7fba8d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.852925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5b44b506-8d9e-40b6-8740-9a559b8323d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6203c822d64dd09651a3683b7fba8d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:09.853027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.853309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.853646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.872245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1129/2235] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:09 2022] DELETE /identity/v3/users/4a6203c822d64dd09651a3683b7fba8d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:09.876415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:09.876706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:09.876978 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:09.882500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] MFA Rules not processed for user `4b895c0c03ae4b398ca24924f3f27dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:09.883701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.897065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:09.899782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.973799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1107/2236] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:09.974980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1130/2237] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:09.986870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.987171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:09.988841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:09.989718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:10.062990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa5cdf-e400-4b0c-a489-5174b9d407e4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:10.066275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f9209-3400-4696-815f-641c5a303925 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:10.068850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:10.069615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:10.070550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:10.070835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:10.072694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:10.073808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:10.074619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:10.075248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:10.083415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:10.086103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:10.087725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:10.089759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:10.097418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1108/2238] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:10.107321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1131/2239] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:05:10.450828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:10.452475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:10.467907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b47775-ed4c-4d19-b331-c3a8b7637f6d service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:10.473566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:10.474471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:10.474762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:10.475038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] PATH_INFO: `/v3/projects/521cd935894f4fd8aba2c88e1fdec534` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:10.476543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=521cd935894f4fd8aba2c88e1fdec534)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:10.479810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:10.530415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521cd935894f4fd8aba2c88e1fdec534', 'request_id': 'req-88cc7c02-6b41-4789-a5ea-ceb28ede5571'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.536155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521cd935894f4fd8aba2c88e1fdec534', 'request_id': 'req-88cc7c02-6b41-4789-a5ea-ceb28ede5571'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.542694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521cd935894f4fd8aba2c88e1fdec534', 'request_id': 'req-88cc7c02-6b41-4789-a5ea-ceb28ede5571'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.554493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] The token cache is being invalidate because project 521cd935894f4fd8aba2c88e1fdec534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:10.554790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.555383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.556950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1109/2240] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:10 2022] DELETE /identity/v3/projects/521cd935894f4fd8aba2c88e1fdec534 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:10.561034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:10.572514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9b2fa-bf0b-4354-93ce-d81b63989f11 service glance] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:10.641061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:10.641928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:10.642238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:10.642542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] PATH_INFO: `/v3/users/723342a7b9ca4a05b528d77d03e83fe2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:10.643721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=723342a7b9ca4a05b528d77d03e83fe2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:10.645626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:10.693648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723342a7b9ca4a05b528d77d03e83fe2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.694306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723342a7b9ca4a05b528d77d03e83fe2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.702185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723342a7b9ca4a05b528d77d03e83fe2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.702864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723342a7b9ca4a05b528d77d03e83fe2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.710446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-717b0725-56b4-4a81-a3ca-5fc5032c52dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723342a7b9ca4a05b528d77d03e83fe2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:10.728970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1132/2241] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:10 2022] DELETE /identity/v3/users/723342a7b9ca4a05b528d77d03e83fe2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:11.005466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.007014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:11.029491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc7c02-6b41-4789-a5ea-ceb28ede5571 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.091018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.091800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.092078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.092345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] PATH_INFO: `/v3/projects/b90053f5060a4ba1a6b9434f675960c0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.093541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90053f5060a4ba1a6b9434f675960c0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.095473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.139521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90053f5060a4ba1a6b9434f675960c0', 'request_id': 'req-01f46c38-7608-40cb-a09a-49df61d797af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:11.144645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90053f5060a4ba1a6b9434f675960c0', 'request_id': 'req-01f46c38-7608-40cb-a09a-49df61d797af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:11.151545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90053f5060a4ba1a6b9434f675960c0', 'request_id': 'req-01f46c38-7608-40cb-a09a-49df61d797af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:11.163156 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] The token cache is being invalidate because project b90053f5060a4ba1a6b9434f675960c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:11.164056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:11.164694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-01f46c38-7608-40cb-a09a-49df61d797af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:11.166373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1110/2242] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:11 2022] DELETE /identity/v3/projects/b90053f5060a4ba1a6b9434f675960c0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:11.171175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.171461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.171737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.189118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:11.190294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:11.247668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1133/2243] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:11.255140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.255429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.255702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.273108 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:11.274282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:11.281997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1111/2244] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:11.286164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.296126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0eb0a-8604-4513-ad28-43b76a06ad12 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.300653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.301292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.301576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.301848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.302599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:11.302903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.305115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.316425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1134/2245] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:11.320563 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.329421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4545d07-80f8-4c68-8f2b-c3aff5d44b0c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.333932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.334595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.334878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.335151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.335824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.337702 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.361907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1112/2246] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/projects => generated 367 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:11.365785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.376649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be2e0-81c5-4b82-999d-7d1c0849889c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.431225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.431893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.432174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.432448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.433145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.435081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.459642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1135/2247] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:11.464030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.474323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f431be-d36a-4cff-b8cc-223736d9182b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.479539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.480190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.480468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.480736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.481447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.483088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.493186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1113/2248] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:05:11.497447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.507464 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eeb3a-881b-4a49-a07c-e24455176f91 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.512102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.512751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.513034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.513311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.514025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.515643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a837b5c-a6f3-4c8c-9d56-a252c452681e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.526087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1136/2249] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:11.530042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.540614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ae88-5a3e-49c2-bfa1-b5207f4c5aec admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.545287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.546166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.546497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.546776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] PATH_INFO: `/v3/projects/8cd26cd8e6b44befad3d613e0f262e8b/users/22c55b2528fe4ae7b9efc221082951c7/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.562968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd26cd8e6b44befad3d613e0f262e8b, user_id=22c55b2528fe4ae7b9efc221082951c7, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.565799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.578267 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1114/2250] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:11 2022] PUT /identity/v3/projects/8cd26cd8e6b44befad3d613e0f262e8b/users/22c55b2528fe4ae7b9efc221082951c7/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:11.589775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.590278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.590377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.615964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] MFA Rules not processed for user `22c55b2528fe4ae7b9efc221082951c7`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:11.617072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:11.678261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1137/2251] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:11.682893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.740290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac27dd-fea4-4ae4-8236-63fe645d13be admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.744914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.745547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.745822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.746091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.747060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.748939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.773583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1115/2252] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:11.777529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.788131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed186a-8095-4b3b-8935-6b4b4c09f349 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.842073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.842768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.843051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.843327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.844028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.846037 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.868808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1138/2253] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:11 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:05:11.869936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.880587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20abe4-feb5-49db-9adb-8db47c990edf admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.885883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.886586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.886867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.887145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.887871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.889466 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.900679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1116/2254] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:11.905662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.916247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2669-b583-4652-8f32-ee90502e192e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:11.920847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:11.921680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:11.921959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:11.922233 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] PATH_INFO: `/v3/projects/cbd5d837b78a44dc953bf92653eaa991/users/de7f2a3ef7084cc98f8e16a7e9c517b3/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:11.936944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd5d837b78a44dc953bf92653eaa991, user_id=de7f2a3ef7084cc98f8e16a7e9c517b3, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:11.939887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:11.953753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1139/2255] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:11 2022] PUT /identity/v3/projects/cbd5d837b78a44dc953bf92653eaa991/users/de7f2a3ef7084cc98f8e16a7e9c517b3/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:11.957818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:11.968383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53948bc0-644e-4de7-b9cb-33b43091b03c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:12.024501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:12.025157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:12.025441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:12.025719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:12.026461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:12.028045 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc742c31-e8df-48af-b093-643ab00d05a9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:12.037213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1117/2256] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:12.041183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:12.051219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda0ca-7e25-4a08-b628-e497a9579200 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:12.056187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:12.057115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:12.057399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:12.057678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] PATH_INFO: `/v3/projects/cbd5d837b78a44dc953bf92653eaa991/users/de7f2a3ef7084cc98f8e16a7e9c517b3/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:12.059819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd5d837b78a44dc953bf92653eaa991, user_id=de7f2a3ef7084cc98f8e16a7e9c517b3, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:12.062546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:12.074475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1140/2257] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:12 2022] PUT /identity/v3/projects/cbd5d837b78a44dc953bf92653eaa991/users/de7f2a3ef7084cc98f8e16a7e9c517b3/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:12.084777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:12.085815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:12.086114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:12.111948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] MFA Rules not processed for user `de7f2a3ef7084cc98f8e16a7e9c517b3`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:12.113233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:12.179869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1118/2258] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:12.190549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:12.192261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:12.247217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2c722-ded0-41b1-a2f6-12382156f341 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:12.250840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:12.251480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:12.251762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:12.252039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:12.261723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:12.263163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3da1f-1c5d-48d2-9065-226056b8999a service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:12.272713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1141/2259] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:13.562374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:13.621018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda280-f017-4d98-8a14-d99686e22f44 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:13.625718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:13.626606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:13.626885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:13.627154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] PATH_INFO: `/v3/users/22c55b2528fe4ae7b9efc221082951c7` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:13.628311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c55b2528fe4ae7b9efc221082951c7)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:13.630232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:13.673550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c55b2528fe4ae7b9efc221082951c7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:13.679895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c55b2528fe4ae7b9efc221082951c7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:13.680541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c55b2528fe4ae7b9efc221082951c7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:13.681134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c55b2528fe4ae7b9efc221082951c7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:13.687905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c55b2528fe4ae7b9efc221082951c7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:13.704339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1119/2260] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:13 2022] DELETE /identity/v3/users/22c55b2528fe4ae7b9efc221082951c7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:13.708751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:13.709216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:13.709309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:13.731319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:13.733252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:13.799240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1142/2261] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:13.808623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:13.810101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:13.866101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500c74-11ec-419a-b9e1-242c7c574981 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:13.869760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:13.870414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:13.870714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:13.870985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:13.880115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:13.881532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:13.891495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1120/2262] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:14.156714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:14.158109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:14.171853 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154b9d2-9b12-4d6e-885d-99aa8344c122 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:14.176754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:14.177608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:14.177881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:14.178155 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] PATH_INFO: `/v3/projects/8cd26cd8e6b44befad3d613e0f262e8b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:14.180433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd26cd8e6b44befad3d613e0f262e8b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:14.182431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:14.236664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd26cd8e6b44befad3d613e0f262e8b', 'request_id': 'req-34eb5dad-bb6f-483d-99f4-3767628352a2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.242808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd26cd8e6b44befad3d613e0f262e8b', 'request_id': 'req-34eb5dad-bb6f-483d-99f4-3767628352a2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.248472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd26cd8e6b44befad3d613e0f262e8b', 'request_id': 'req-34eb5dad-bb6f-483d-99f4-3767628352a2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.261443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] The token cache is being invalidate because project 8cd26cd8e6b44befad3d613e0f262e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:14.261730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.262358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.264073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1143/2263] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:14 2022] DELETE /identity/v3/projects/8cd26cd8e6b44befad3d613e0f262e8b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:14.268072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:14.277973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad20-7a8f-4417-bd2a-f42252c93d52 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:14.338433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:14.339374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:14.339659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:14.339932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] PATH_INFO: `/v3/users/de7f2a3ef7084cc98f8e16a7e9c517b3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:14.341105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7f2a3ef7084cc98f8e16a7e9c517b3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:14.343042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:14.390877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7f2a3ef7084cc98f8e16a7e9c517b3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.399210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7f2a3ef7084cc98f8e16a7e9c517b3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.399956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7f2a3ef7084cc98f8e16a7e9c517b3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.400559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7f2a3ef7084cc98f8e16a7e9c517b3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.407848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-12de0449-771e-479a-b09e-40a80c830243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7f2a3ef7084cc98f8e16a7e9c517b3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.424474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1121/2264] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:14 2022] DELETE /identity/v3/users/de7f2a3ef7084cc98f8e16a7e9c517b3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:14.766306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:14.767768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:14.789442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb5dad-bb6f-483d-99f4-3767628352a2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:14.849714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:14.851137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:14.851419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:14.851691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] PATH_INFO: `/v3/projects/cbd5d837b78a44dc953bf92653eaa991` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:14.852864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd5d837b78a44dc953bf92653eaa991)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:14.854808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:14.900543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd5d837b78a44dc953bf92653eaa991', 'request_id': 'req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.906659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd5d837b78a44dc953bf92653eaa991', 'request_id': 'req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.912036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd5d837b78a44dc953bf92653eaa991', 'request_id': 'req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.927712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] The token cache is being invalidate because project cbd5d837b78a44dc953bf92653eaa991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:14.927944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.928557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-a3e5302c-c978-41c3-b1ee-b0f8d055ea3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:14.930698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1144/2265] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:14 2022] DELETE /identity/v3/projects/cbd5d837b78a44dc953bf92653eaa991 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:14.935728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:14.936286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:14.936377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:14.960224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:14.961561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:15.023111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1122/2266] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:15.031722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.032013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.032293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.050368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:15.051574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:15.060120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1145/2267] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:15.064351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.074332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da91d0-5771-4c5c-98ae-120d8efa483e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.079088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.079741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.080056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.080333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.081084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:15.081390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.083039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.096762 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1123/2268] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:15.102166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.111941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae5671-0643-4502-9bac-095b3f5f3a82 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.117624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.118363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.118667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.118950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.119687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.122105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.155347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1146/2269] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:15.156586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.167647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f2f34-13ae-4201-8e8a-57b196e0ad9a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.222716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.223381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.223663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.223941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.224653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.227288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.246009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1124/2270] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:15.255675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.266095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91eb612-efab-42d0-908e-0f0f74275df8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.272327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.273706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.273991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.274268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.275041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.276870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.287954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1147/2271] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:15.292448 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.302074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9ace3-7871-44fe-9b1e-c8f0fd80fbed admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.306664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.307308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.307585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.307856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.308566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.310138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc782e-f5f6-4019-bed0-489d0af5d8db admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.320061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1125/2272] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:15.323970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.333642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2a97e-5117-4426-9151-df46b2c8ca47 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.338738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.339599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.339879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.340154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] PATH_INFO: `/v3/projects/df97fc203f714fba98fe69d4f8de4fc3/users/e9d086b612fb43d29e585f9a8f01d310/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.355788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=df97fc203f714fba98fe69d4f8de4fc3, user_id=e9d086b612fb43d29e585f9a8f01d310, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.358487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.372333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1148/2273] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:15 2022] PUT /identity/v3/projects/df97fc203f714fba98fe69d4f8de4fc3/users/e9d086b612fb43d29e585f9a8f01d310/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:05:15.382632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.382950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.383233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.410484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] MFA Rules not processed for user `e9d086b612fb43d29e585f9a8f01d310`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:15.411710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:15.485946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1126/2274] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:15.491569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.576070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bd725-cb48-4ef3-a7cc-d4d99766b5aa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.581765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.582585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.582932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.583210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.583905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.586571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.619821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1149/2275] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:15.623971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.635815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb65459-bba5-400b-8802-e1cad355bf1e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.701600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.702625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.702913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.703188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.703897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.705808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.731188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1127/2276] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:15.734534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.745368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b33941-5539-4748-8298-9549c8c84ddf admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.750673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.751832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.752264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.752705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.753572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.755328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.766564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1150/2277] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:05:15.770423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.780541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5132844-65b9-40ad-921e-fd96028828f7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.785273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.785938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.786220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.786542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.787265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.789405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e94ff-b69c-4b06-9352-e15f881c1a7f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.799594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1128/2278] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:15.804997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:15.816061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760a4a-f266-4bf1-b560-e72809447962 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:15.821346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:15.822275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.822634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.822914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] PATH_INFO: `/v3/projects/48d560f478f74887a33ab22d5de66869/users/a9dc554da317475dbb52adb72cbf7d5c/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.837953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d560f478f74887a33ab22d5de66869, user_id=a9dc554da317475dbb52adb72cbf7d5c, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:15.840631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:15.861900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1151/2279] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:15 2022] PUT /identity/v3/projects/48d560f478f74887a33ab22d5de66869/users/a9dc554da317475dbb52adb72cbf7d5c/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:15.873399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:15.873689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:15.873964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:15.903561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] MFA Rules not processed for user `a9dc554da317475dbb52adb72cbf7d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:15.904760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:15.967334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1129/2280] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:15.971842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.031080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842d09d-0e42-475b-8b88-b3167aaedb18 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.035756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.036399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.036754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.037028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.037725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.040273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.064423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1152/2281] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:15 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:16.068689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.080248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01bccf-d6e8-4fe6-a960-8a03f44296a5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.146994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.148238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.148238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.149224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.149971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.151918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.175998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1130/2282] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:16 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:16.177492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.187802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf77f67-ca5c-4ad7-b6f9-aa0b257dfaec admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.192531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.193190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.193469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.193743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.194550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.196135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.207901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1153/2283] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:16.212764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.224811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d99551-38d1-4bcd-bdfa-e81050377ee5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.230769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.231673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.231950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.232220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] PATH_INFO: `/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96/users/6539e45ef003497cb02ef57540be45f5/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.248502 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e0cd25e5ce4cda8add856fada8ec96, user_id=6539e45ef003497cb02ef57540be45f5, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.251163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.264055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1131/2284] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:16 2022] PUT /identity/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96/users/6539e45ef003497cb02ef57540be45f5/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:16.267900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.278326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3cffd-32bb-4aa6-9b6e-90366693ad60 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.331406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.331994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.332277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.332553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.333269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.334918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa9587-73cb-4aa6-a1dd-81598bb0cfb8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.344859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1154/2285] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:16.348928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.358781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55da143-de72-4437-8e79-cde4aa04ea6a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.363288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.364145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.364423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.364694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] PATH_INFO: `/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96/users/6539e45ef003497cb02ef57540be45f5/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.366663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e0cd25e5ce4cda8add856fada8ec96, user_id=6539e45ef003497cb02ef57540be45f5, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.369298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.380199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1132/2286] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:16 2022] PUT /identity/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96/users/6539e45ef003497cb02ef57540be45f5/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:16.393415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.393938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.394525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.421435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] MFA Rules not processed for user `6539e45ef003497cb02ef57540be45f5`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:16.422660 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:16.485591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1155/2287] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:16.496058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:16.497975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:16.552091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2e791-b1d3-4eff-9d49-6d22dea8996a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:16.555669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:16.556417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:16.556802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:16.557237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:16.567118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:16.568555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:16.578135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1133/2288] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:17.103004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:17.104388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:17.117280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2349f-3b64-4898-896b-fa4029e4d882 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:17.120927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:17.121905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:17.122192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:17.122552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:17.177819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:17.179898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc6198-4348-4e84-8639-f522ab743b4e service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:17.188545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1156/2289] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:20.057673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:20.059237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:20.122413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820f3a6-56a2-4636-aabd-3e39004fd019 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:20.127459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:20.128405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:20.128683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:20.128955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] PATH_INFO: `/v3/users/4dcc55c90ec84e7f974e95bf183bc6eb` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:20.130110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dcc55c90ec84e7f974e95bf183bc6eb)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:20.132064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:20.175568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dcc55c90ec84e7f974e95bf183bc6eb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.182666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dcc55c90ec84e7f974e95bf183bc6eb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.183340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dcc55c90ec84e7f974e95bf183bc6eb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.183963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dcc55c90ec84e7f974e95bf183bc6eb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.190352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dcc55c90ec84e7f974e95bf183bc6eb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.206467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1134/2290] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:20 2022] DELETE /identity/v3/users/4dcc55c90ec84e7f974e95bf183bc6eb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:20.210929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:20.211383 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:20.211472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:20.228434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:20.229622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:20.285393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1157/2291] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:20.295621 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:20.297564 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:20.361423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fa49-e815-48b0-b0a7-70d9c1be2783 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:20.365523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:20.366756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:20.367055 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:20.367328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:20.377119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:20.378598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:20.388400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1135/2292] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:05:20.736032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:20.737581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:20.750977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87010ce-f9b3-48f7-a9fb-ceb72ba1df00 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:20.756023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:20.757192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:20.757876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:20.758156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] PATH_INFO: `/v3/projects/5592661152b24accba1b0a108519dcde` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:20.760822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5592661152b24accba1b0a108519dcde)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:20.762901 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:20.818694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5592661152b24accba1b0a108519dcde', 'request_id': 'req-764723f1-276b-4316-8abf-04a44fb19aca'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.827462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5592661152b24accba1b0a108519dcde', 'request_id': 'req-764723f1-276b-4316-8abf-04a44fb19aca'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.834176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5592661152b24accba1b0a108519dcde', 'request_id': 'req-764723f1-276b-4316-8abf-04a44fb19aca'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.851641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] The token cache is being invalidate because project 5592661152b24accba1b0a108519dcde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:20.852080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.852965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:20.855215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1158/2293] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:20 2022] DELETE /identity/v3/projects/5592661152b24accba1b0a108519dcde => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:05:21.242006 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:21.244658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:21.301399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16168854-a765-4384-89df-bf6c8f38c23d service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:21.349387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:21.350034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:21.350319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:21.351231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:21.353011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:21.411281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:21.412713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26dd3-b5c0-4de5-9dda-432d19a47444 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:21.422881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1136/2294] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:23.674657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:23.676242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:23.744780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764723f1-276b-4316-8abf-04a44fb19aca admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:23.751083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:23.751970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:23.752250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:23.752524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] PATH_INFO: `/v3/users/e9d086b612fb43d29e585f9a8f01d310` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:23.756480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d086b612fb43d29e585f9a8f01d310)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:23.759065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:23.814342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d086b612fb43d29e585f9a8f01d310'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:23.814999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d086b612fb43d29e585f9a8f01d310'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:23.821488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d086b612fb43d29e585f9a8f01d310'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:23.822100 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d086b612fb43d29e585f9a8f01d310'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:23.829288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d086b612fb43d29e585f9a8f01d310'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:23.847191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1159/2295] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:23 2022] DELETE /identity/v3/users/e9d086b612fb43d29e585f9a8f01d310 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:23.852299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22cc4-c43e-4427-a4b2-c9b2b958dd1f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:23.852825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22cc4-c43e-4427-a4b2-c9b2b958dd1f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:23.852950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22cc4-c43e-4427-a4b2-c9b2b958dd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:23.871276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4da22cc4-c43e-4427-a4b2-c9b2b958dd1f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:23.872710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4da22cc4-c43e-4427-a4b2-c9b2b958dd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:23.929252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1137/2296] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:23.938728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:23.940180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:24.003133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd8c0-54fb-41b2-a8b7-051ebd9db99c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:24.006947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:24.007626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.007905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.008178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.020160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:24.021617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:24.031656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1160/2297] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:24.090790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.091147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.091486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.120436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:24.121955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:24.196948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1138/2298] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:24.241603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:24.243350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:24.317435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7cc33-0cf4-4f31-a10d-026b194af3c4 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:24.323002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:24.323652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.323933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.324208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.326855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:24.328260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:24.333028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:24.334629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:24.344170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1161/2299] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:05:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:05:24.345767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a7664-1f84-4e59-a476-e5c861646c6b None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:24.351959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:24.352996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.353413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.353698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] PATH_INFO: `/v3/projects/df97fc203f714fba98fe69d4f8de4fc3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.355734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df97fc203f714fba98fe69d4f8de4fc3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:24.357680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:24.407352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df97fc203f714fba98fe69d4f8de4fc3', 'request_id': 'req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.412697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df97fc203f714fba98fe69d4f8de4fc3', 'request_id': 'req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.420612 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df97fc203f714fba98fe69d4f8de4fc3', 'request_id': 'req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.436527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] The token cache is being invalidate because project df97fc203f714fba98fe69d4f8de4fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:24.436824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.437506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-9f6f9087-f2e9-4eab-bb3d-36c38afe3249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.439418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1139/2300] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:24 2022] DELETE /identity/v3/projects/df97fc203f714fba98fe69d4f8de4fc3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:24.445987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:24.465213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2ed7-976d-49ca-a1ca-73e4495ab045 service swift] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:24.504039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b46ac-770d-4bd7-b28f-cd6675b8aa48 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.504330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b46ac-770d-4bd7-b28f-cd6675b8aa48 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.504610 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b46ac-770d-4bd7-b28f-cd6675b8aa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.527323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e11b46ac-770d-4bd7-b28f-cd6675b8aa48 None None] MFA Rules not processed for user `3bd41b34f5b14ae0a0ebccec9d59944f`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:24.528606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e11b46ac-770d-4bd7-b28f-cd6675b8aa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:24.550843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:24.553941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.554225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.554531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] PATH_INFO: `/v3/users/a9dc554da317475dbb52adb72cbf7d5c` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.556571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9dc554da317475dbb52adb72cbf7d5c)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:24.558377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:24.608275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1140/2301] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:24.657352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9dc554da317475dbb52adb72cbf7d5c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.658249 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9dc554da317475dbb52adb72cbf7d5c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.668798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9dc554da317475dbb52adb72cbf7d5c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.669997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9dc554da317475dbb52adb72cbf7d5c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.684061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9dc554da317475dbb52adb72cbf7d5c'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:24.712382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1162/2302] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:24 2022] DELETE /identity/v3/users/a9dc554da317475dbb52adb72cbf7d5c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:24.978107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:24.978622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:24.978733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:24.998049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:24.999487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.060511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1141/2303] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:25.083847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:25.094452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.176940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94259b4-2955-43fd-ba12-405c1205181e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:25.186096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:25.186836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:25.187120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:25.187391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:25.201585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:25.203228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7ec02-a9ec-4844-b11c-ec0a004807cb service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:25.212991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1163/2304] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:05:25.340854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:25.343477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.466620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cbd89-1288-4d92-8940-7b8808452d1f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:25.472508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:25.473375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:25.473653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:25.474092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] PATH_INFO: `/v3/projects/48d560f478f74887a33ab22d5de66869` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:25.477615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d560f478f74887a33ab22d5de66869)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:25.479589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:25.487540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:25.487918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:25.488266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:25.509930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:25.511174 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.523020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1164/2305] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:25.536436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:25.539266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d560f478f74887a33ab22d5de66869', 'request_id': 'req-543493b9-28aa-4290-bc1f-cea06ff5d49b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.545311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d560f478f74887a33ab22d5de66869', 'request_id': 'req-543493b9-28aa-4290-bc1f-cea06ff5d49b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.553510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d560f478f74887a33ab22d5de66869', 'request_id': 'req-543493b9-28aa-4290-bc1f-cea06ff5d49b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.559523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fca0c-3c0e-4228-bf8a-f21170e78a37 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:25.572341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] The token cache is being invalidate because project 48d560f478f74887a33ab22d5de66869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:25.572878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.574014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.578465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1142/2306] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:25 2022] DELETE /identity/v3/projects/48d560f478f74887a33ab22d5de66869 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:25.581837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:25.584073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.641446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:25.642101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:25.642594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:25.643027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:25.645026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:25.672398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543493b9-28aa-4290-bc1f-cea06ff5d49b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:25.681159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:25.682964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:25.683521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:25.684142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] PATH_INFO: `/v3/users/6539e45ef003497cb02ef57540be45f5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:25.686770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6539e45ef003497cb02ef57540be45f5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:25.689835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:25.718936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:25.720724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:25.731033 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1165/2307] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:05:25.746920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6539e45ef003497cb02ef57540be45f5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.754659 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6539e45ef003497cb02ef57540be45f5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.755422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6539e45ef003497cb02ef57540be45f5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.756249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6539e45ef003497cb02ef57540be45f5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.765372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-a8ced27a-31ba-4066-9bbc-2324a748b70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6539e45ef003497cb02ef57540be45f5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:25.787144 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1143/2308] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:25 2022] DELETE /identity/v3/users/6539e45ef003497cb02ef57540be45f5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:26.152173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.168855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.169364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.169472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.172390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650be8-53f5-4761-b571-14f1239f7f40 service swift] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.194898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:26.196136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.238680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.239658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.240207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.240550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] PATH_INFO: `/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.242028 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e0cd25e5ce4cda8add856fada8ec96)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.244336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.257830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1144/2309] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:26.270342 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.271790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.289926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e0cd25e5ce4cda8add856fada8ec96', 'request_id': 'req-d51dc643-e554-493a-9bb2-309ca4989ab3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:26.296416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e0cd25e5ce4cda8add856fada8ec96', 'request_id': 'req-d51dc643-e554-493a-9bb2-309ca4989ab3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:26.301737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e0cd25e5ce4cda8add856fada8ec96', 'request_id': 'req-d51dc643-e554-493a-9bb2-309ca4989ab3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:26.313032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] The token cache is being invalidate because project c3e0cd25e5ce4cda8add856fada8ec96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:26.313366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:26.313979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d51dc643-e554-493a-9bb2-309ca4989ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:26.315553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1166/2310] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:26 2022] DELETE /identity/v3/projects/c3e0cd25e5ce4cda8add856fada8ec96 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:26.322159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0f1b-7020-44e5-b59e-d2df524aa107 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.322478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0f1b-7020-44e5-b59e-d2df524aa107 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.322765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0f1b-7020-44e5-b59e-d2df524aa107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.339818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e1a55-c258-48f5-bec8-1a3011b911ac None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.340522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-feac0f1b-7020-44e5-b59e-d2df524aa107 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:26.341711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-feac0f1b-7020-44e5-b59e-d2df524aa107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.399404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.399755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.400039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.400305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.401627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1167/2311] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:26.402595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.411163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.411920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.412393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.437877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:26.439114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.448153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1168/2312] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:26.452252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.461109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2343d-353f-408a-812e-afca8f9a6134 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.465815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.466499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.466773 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.467046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.467201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.467882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:26.468188 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.469014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.469752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.478111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1145/2313] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:05:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 15:05:26.481935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1169/2314] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:05:26.486672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.500667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a145d-2350-4c49-9556-d5ec7292c29b service swift] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.506136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.507398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.507685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.508256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.509131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.511117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.543329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1146/2315] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:26.547884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.561629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ed8a0-6e07-46e3-9d66-ebd2fa979ce3 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.637599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.638330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.638656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.638998 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.639750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.641742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.669508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1170/2316] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:26.671376 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.685445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047dc44-ea43-419d-ba8c-77bfd82305db admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.692508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.694022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.694899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.695395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.697467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.702475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.720862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1147/2317] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:26.727080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.740151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299ec2-4ff8-40eb-86bd-1da5e3577e4f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.751943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.751943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.751943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.751943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.752809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.755459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f0786-3e0b-4455-9ffc-6a1999060e50 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.773665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1171/2318] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:26.786340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:26.804100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c6156-e826-407f-af86-4d1f6697e7b6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:26.810756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:26.812061 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.812451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.812830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] PATH_INFO: `/v3/projects/ca670514821a41e6858e25f14495936d/users/6a9952d7ccf14564a09f023e8e60dcd2/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.836131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca670514821a41e6858e25f14495936d, user_id=6a9952d7ccf14564a09f023e8e60dcd2, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:26.842456 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:26.871660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1148/2319] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:26 2022] PUT /identity/v3/projects/ca670514821a41e6858e25f14495936d/users/6a9952d7ccf14564a09f023e8e60dcd2/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:26.874813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:26.875428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:26.875854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:26.910087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] MFA Rules not processed for user `6a9952d7ccf14564a09f023e8e60dcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:26.910087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:26.997482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1172/2320] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:26.999535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.065697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fd8fc-b839-47c5-a8ba-a0f719343a83 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.075447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.076107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.076390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.076661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.077346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.079410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.115985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1149/2321] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:26 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:27.120373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.131161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1dc23-0103-4363-90f4-d4ea3b783cb5 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.202500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.203380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.203778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.204137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.204993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.207396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.239893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1173/2322] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:27 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:27.244624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.256326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cbc7-03d8-46cd-9ad5-056478745405 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.261923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.262786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.263094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.263435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.264252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.267162 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.282615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1150/2323] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:27.286638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.302701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b00791-7e6b-467c-b32b-45aaea6292f1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.307599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.308423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.308770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.310304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.310304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.312094 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177db135-a32c-420d-9a0a-06fc5a3738ee admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.324899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1174/2324] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:27.328902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.340611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d6394-a1dc-4538-a073-4382b2a57966 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.346080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.347311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.347738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.348180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] PATH_INFO: `/v3/projects/6f3347242dd044a591a2a8c734310452/users/80a6277673ae4fb98b5846f7b534b546/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.366192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3347242dd044a591a2a8c734310452, user_id=80a6277673ae4fb98b5846f7b534b546, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.369679 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.418291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1151/2325] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:27 2022] PUT /identity/v3/projects/6f3347242dd044a591a2a8c734310452/users/80a6277673ae4fb98b5846f7b534b546/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:05:27.429124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.429421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.430302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.484064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] MFA Rules not processed for user `80a6277673ae4fb98b5846f7b534b546`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:27.485306 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:27.572525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1175/2326] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:27.578941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.684372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30628e9-7c46-4f27-8d20-1180debb9071 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.697798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.702763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.702763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.702763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.702763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.702763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.743845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1152/2327] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:27 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:27.744659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.757439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170251d0-03dc-4d1f-bfab-a19018d3ef13 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.827456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.828076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.828535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.828839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.829598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.831642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.853527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1176/2328] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:27 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:27.857775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.869085 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe13b-0d63-45f8-a944-71859cebd4e3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.874835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.875512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.875820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.876068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.876824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.879189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.893842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1153/2329] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:27.898276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.911057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491256d-a9b0-48a8-a257-451cb873ebc4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:27.916535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:27.917438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:27.918025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:27.918317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] PATH_INFO: `/v3/projects/ab6e10cb3b3440348a34b41d53d6620c/users/3de0d52ba4be464b88d01c4d87229d23/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:27.935802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6e10cb3b3440348a34b41d53d6620c, user_id=3de0d52ba4be464b88d01c4d87229d23, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:27.939730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:27.960134 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1177/2330] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:27 2022] PUT /identity/v3/projects/ab6e10cb3b3440348a34b41d53d6620c/users/3de0d52ba4be464b88d01c4d87229d23/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:27.963844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:27.973580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df16f3-d450-4bcc-bbd6-5f061c3792f8 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:28.029654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:28.030319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:28.030626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:28.030903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:28.031618 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:28.033208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e956379-7aa0-4f5f-9f9a-b755c6b8659f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:28.042827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1154/2331] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:28.046842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:28.057352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f426e-3f77-4d17-83eb-b1b52aa12ed7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:28.063942 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:28.064822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:28.065849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:28.065849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] PATH_INFO: `/v3/projects/ab6e10cb3b3440348a34b41d53d6620c/users/3de0d52ba4be464b88d01c4d87229d23/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:28.067431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6e10cb3b3440348a34b41d53d6620c, user_id=3de0d52ba4be464b88d01c4d87229d23, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:28.070886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:28.081794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1178/2332] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:28 2022] PUT /identity/v3/projects/ab6e10cb3b3440348a34b41d53d6620c/users/3de0d52ba4be464b88d01c4d87229d23/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:28.092870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:28.093146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:28.093449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:28.121064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] MFA Rules not processed for user `3de0d52ba4be464b88d01c4d87229d23`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:28.122587 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:28.189839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1155/2333] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:28.201916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:28.203702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:28.291342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b5438-375e-41a5-a324-4541dcc195de admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:28.295894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:28.296521 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:28.296816 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:28.297081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:28.306700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:28.308127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:28.317833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1179/2334] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:28.367154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:28.368604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:28.380459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227273-73ea-4587-94bf-fb9d5a8f6a73 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:28.384999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:28.385638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:28.385917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:28.386187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:28.463405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:28.465508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a23c6-25d0-46cc-8574-8c52662d115b service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:28.478697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1156/2335] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:28.977025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:28.978779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.050744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2b97-484d-4298-9c82-d103c1f427f4 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.056217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.057315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.057594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.057907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] PATH_INFO: `/v3/users/327ee22061174279a140e12445039b60` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.060473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327ee22061174279a140e12445039b60)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.062766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.107040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327ee22061174279a140e12445039b60'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.107676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327ee22061174279a140e12445039b60'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.114021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327ee22061174279a140e12445039b60'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.114664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327ee22061174279a140e12445039b60'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.124333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327ee22061174279a140e12445039b60'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.140892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1180/2336] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:28 2022] DELETE /identity/v3/users/327ee22061174279a140e12445039b60 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:29.145379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.145673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.145953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.163352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:29.164540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.219641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1157/2337] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:29.229743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.231145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.297136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac829c-7017-4ad3-82b6-0b1330fcbe45 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.300817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.301662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.302065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.302343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.312424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.314301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475c19d-11d8-427b-b0ec-17a9af94037f service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.328372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1181/2338] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:29.492330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.493868 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.507780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913250c8-277d-4249-aa57-9e37a418e7ad None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.512940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.513849 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.514164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.514468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] PATH_INFO: `/v3/projects/2954db13c70c43b98c05d392089b3a93` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.515657 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2954db13c70c43b98c05d392089b3a93)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.517605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.563351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2954db13c70c43b98c05d392089b3a93', 'request_id': 'req-cd359102-5b49-408e-ba4b-262ed9706892'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.568433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2954db13c70c43b98c05d392089b3a93', 'request_id': 'req-cd359102-5b49-408e-ba4b-262ed9706892'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.574320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2954db13c70c43b98c05d392089b3a93', 'request_id': 'req-cd359102-5b49-408e-ba4b-262ed9706892'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.587817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] The token cache is being invalidate because project 2954db13c70c43b98c05d392089b3a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:29.588103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.588716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cd359102-5b49-408e-ba4b-262ed9706892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:29.590265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1158/2339] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:29 2022] DELETE /identity/v3/projects/2954db13c70c43b98c05d392089b3a93 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:05:29.595449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.595807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.596154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.614957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:29.616267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.683107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1182/2340] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:29.691362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.691652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.691931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.711631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:29.713082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:29.729704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1159/2341] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:29.730937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.740642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c60e0-475c-4599-9c2e-1617a763cc3a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.745263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.745914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.746195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.746844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.747559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:29.747859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.749446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.765601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1183/2342] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:29.769992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.784258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d6eaa-728e-4d53-bb88-3fefafdbc8d5 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.789114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.789796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.790078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.790355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.791201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.793094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.819649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1160/2343] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:29 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:29.821328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.830869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7a343-aabc-4ea0-9009-57b4b63ebda7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.885579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.886257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.886744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.887031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.887757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.889675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.919316 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1184/2344] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:29 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:29.919501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.930275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49433d-e6d3-4431-b735-878dbf283852 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.935535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.936228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.936472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.936767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.937480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.939337 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.956958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1161/2345] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:05:29.961192 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:29.971599 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a735d47-f188-4f44-be0a-71ec7115b9fe admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:29.976262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:29.976918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:29.977195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:29.977739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:29.978171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:29.979766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608003c-cb77-49b3-b0f2-09ec907eaffd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:29.991688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1185/2346] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:29.995795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:30.006333 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878253-c9de-4acb-a410-4cc133f8bcb5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:30.011998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:30.012907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:30.013185 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:30.013482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] PATH_INFO: `/v3/projects/8e1c094fcd8a478db9b0685e7db8f0db/users/400559e7c47e4c0e94b2a539fcbd515e/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:30.029041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1c094fcd8a478db9b0685e7db8f0db, user_id=400559e7c47e4c0e94b2a539fcbd515e, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:30.031700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:30.053945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1162/2347] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:29 2022] PUT /identity/v3/projects/8e1c094fcd8a478db9b0685e7db8f0db/users/400559e7c47e4c0e94b2a539fcbd515e/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:30.059357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:30.059534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:30.059829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:30.092549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] MFA Rules not processed for user `400559e7c47e4c0e94b2a539fcbd515e`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:30.094153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:30.175361 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1186/2348] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:30.188656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:30.190830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:30.259354 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae840-3ecd-4be4-a894-b4c801f2298b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:30.263361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:30.264015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:30.264299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:30.264575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:30.273937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:30.275360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9d893-d697-43f2-8b10-07d11b0fe362 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:30.285482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1163/2349] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:31.402065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:31.403632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:31.466308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a59c13-b990-488c-bc95-edaee7f255ff None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:31.470642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:31.471502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:31.471780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:31.472052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] PATH_INFO: `/v3/users/6a9952d7ccf14564a09f023e8e60dcd2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:31.473221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9952d7ccf14564a09f023e8e60dcd2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:31.475170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:31.523967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9952d7ccf14564a09f023e8e60dcd2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:31.524983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9952d7ccf14564a09f023e8e60dcd2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:31.532560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9952d7ccf14564a09f023e8e60dcd2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:31.533292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9952d7ccf14564a09f023e8e60dcd2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:31.540839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9952d7ccf14564a09f023e8e60dcd2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:31.558382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1187/2350] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:31 2022] DELETE /identity/v3/users/6a9952d7ccf14564a09f023e8e60dcd2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:31.563820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:31.564306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:31.564893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:31.582822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:31.584013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:31.648925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1164/2351] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:31.660029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:31.662288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:31.720030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bef7a-cebc-4875-9b3d-40c3058f4452 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:31.723581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:31.724227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:31.724504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:31.724775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:31.733698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:31.735278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:31.749611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1188/2352] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:32.034519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:32.036035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:32.049476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb3c2e-2f71-457c-8cb0-46d08a46a75f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:32.060572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:32.061792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:32.062084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:32.062596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] PATH_INFO: `/v3/projects/ca670514821a41e6858e25f14495936d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:32.064341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca670514821a41e6858e25f14495936d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:32.066305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:32.111468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca670514821a41e6858e25f14495936d', 'request_id': 'req-5615d370-aff8-4ac1-904f-7fa8e74db298'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.117491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca670514821a41e6858e25f14495936d', 'request_id': 'req-5615d370-aff8-4ac1-904f-7fa8e74db298'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.123893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca670514821a41e6858e25f14495936d', 'request_id': 'req-5615d370-aff8-4ac1-904f-7fa8e74db298'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.136630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] The token cache is being invalidate because project ca670514821a41e6858e25f14495936d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:32.137064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.137681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.139277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1165/2353] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:32 2022] DELETE /identity/v3/projects/ca670514821a41e6858e25f14495936d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:32.143198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:32.152730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff89f1-66e3-4daf-9ead-4848c02fd8b2 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:32.208576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:32.209523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:32.209836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:32.210114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] PATH_INFO: `/v3/users/80a6277673ae4fb98b5846f7b534b546` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:32.211324 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a6277673ae4fb98b5846f7b534b546)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:32.213234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:32.258056 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a6277673ae4fb98b5846f7b534b546'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.258775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a6277673ae4fb98b5846f7b534b546'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.265969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a6277673ae4fb98b5846f7b534b546'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.266727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a6277673ae4fb98b5846f7b534b546'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.274475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a6277673ae4fb98b5846f7b534b546'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.291378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1189/2354] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:32 2022] DELETE /identity/v3/users/80a6277673ae4fb98b5846f7b534b546 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:32.546633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:32.548126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:32.561981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d370-aff8-4ac1-904f-7fa8e74db298 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:32.617768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:32.618781 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:32.619087 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:32.619379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] PATH_INFO: `/v3/projects/6f3347242dd044a591a2a8c734310452` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:32.620547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3347242dd044a591a2a8c734310452)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:32.622771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:32.672580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3347242dd044a591a2a8c734310452', 'request_id': 'req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.677716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3347242dd044a591a2a8c734310452', 'request_id': 'req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.683730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3347242dd044a591a2a8c734310452', 'request_id': 'req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.696872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] The token cache is being invalidate because project 6f3347242dd044a591a2a8c734310452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:32.697154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.697768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.699353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1166/2355] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:32 2022] DELETE /identity/v3/projects/6f3347242dd044a591a2a8c734310452 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:32.703634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:32.705145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:32.718285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d4be-2402-452e-9679-c0b2456903ae admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:32.787944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:32.788878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:32.789239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:32.789604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] PATH_INFO: `/v3/users/3de0d52ba4be464b88d01c4d87229d23` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:32.790873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de0d52ba4be464b88d01c4d87229d23)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:32.792896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:32.836270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de0d52ba4be464b88d01c4d87229d23'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.836920 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de0d52ba4be464b88d01c4d87229d23'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.843506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de0d52ba4be464b88d01c4d87229d23'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.844120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de0d52ba4be464b88d01c4d87229d23'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.850866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f351fe9-0963-4ad9-a3e0-d729fb0c3864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de0d52ba4be464b88d01c4d87229d23'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:32.867284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1190/2356] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:32 2022] DELETE /identity/v3/users/3de0d52ba4be464b88d01c4d87229d23 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:33.124103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.125554 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:33.140695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2d6d-6629-476f-9c30-9c9908b4caaf admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.194410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.195283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.195561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.195835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] PATH_INFO: `/v3/projects/ab6e10cb3b3440348a34b41d53d6620c` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.196983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6e10cb3b3440348a34b41d53d6620c)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.198919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.243869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6e10cb3b3440348a34b41d53d6620c', 'request_id': 'req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:33.249405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6e10cb3b3440348a34b41d53d6620c', 'request_id': 'req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:33.255422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6e10cb3b3440348a34b41d53d6620c', 'request_id': 'req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:33.268168 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] The token cache is being invalidate because project ab6e10cb3b3440348a34b41d53d6620c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:33.268632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:33.269316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-606ffae6-b98b-45b3-94e8-2cea19d3ccaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:33.271019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1167/2357] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:33 2022] DELETE /identity/v3/projects/ab6e10cb3b3440348a34b41d53d6620c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:33.277394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.277682 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.277955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.297217 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:33.298443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:33.355715 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1191/2358] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:33.365016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.365381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.365757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.385874 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:33.387180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:33.395338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1168/2359] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:33.400214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.409756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0be2f-5520-44b8-8ef6-330c4ad5c2e2 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.414939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.415608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.415892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.416171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.416884 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:33.417189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.418798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.429803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1192/2360] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:33.433965 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.442919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52547038-3ab4-4bd2-be3a-f7b6444d75ae None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.447439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.448077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.448356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.448631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.449305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.451248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.475978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1169/2361] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:33 2022] POST /identity/v3/projects => generated 373 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:33.480067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.489594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1db-bd4d-4d75-bb4f-9faf4b829c76 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.545819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.546498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.546784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.547057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.547733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.549636 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.571290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1193/2362] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:33 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:05:33.575332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.586211 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd74a8-a4a9-4989-9fff-36a6f5b26de7 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.590938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.591584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.591866 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.592139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.592853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.594472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.605234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1170/2363] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:33.609334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.619836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2178ac-7ae7-4285-8a4d-5c974a6dde90 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.624794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.625459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.625745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.626023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.626788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.628380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae902fa4-f2ac-4efa-8fdd-b223b3fa6172 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.638754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1194/2364] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:33.643031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.653542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6870b-71fd-4055-977a-e8a413596e06 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.658128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.659025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.659305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.659575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] PATH_INFO: `/v3/projects/9e243222065440a89e7af5c5ad8a295f/users/9bd03939c50b46ceb80cbfc82329e6c3/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.673164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e243222065440a89e7af5c5ad8a295f, user_id=9bd03939c50b46ceb80cbfc82329e6c3, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.675836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.689914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1171/2365] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:33 2022] PUT /identity/v3/projects/9e243222065440a89e7af5c5ad8a295f/users/9bd03939c50b46ceb80cbfc82329e6c3/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:33.701527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.701841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.702115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.728079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] MFA Rules not processed for user `9bd03939c50b46ceb80cbfc82329e6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:33.729266 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:33.789534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1195/2366] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:33.801303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.803193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:33.877263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb29356-ebd6-448a-8c4a-de3b078c0537 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.881033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.882346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.882652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.883112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.891497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:33.892929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27e162-e24f-4614-b2fe-0200591685c0 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:33.901851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1172/2367] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:33.929107 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:33.989765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655aec-d40a-4837-b2f5-39741896afc0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:33.996111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:33.997209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:33.997427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:33.997705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] PATH_INFO: `/v3/users/9bd03939c50b46ceb80cbfc82329e6c3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:33.998934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd03939c50b46ceb80cbfc82329e6c3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:34.000851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:34.045563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd03939c50b46ceb80cbfc82329e6c3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.046232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd03939c50b46ceb80cbfc82329e6c3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.052692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd03939c50b46ceb80cbfc82329e6c3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.053313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd03939c50b46ceb80cbfc82329e6c3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.059471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd03939c50b46ceb80cbfc82329e6c3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.077387 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1196/2368] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:33 2022] DELETE /identity/v3/users/9bd03939c50b46ceb80cbfc82329e6c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:34.082147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:34.082457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:34.082732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:34.102150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:34.103391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:34.159220 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1173/2369] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:34.169323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:34.170784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:34.224022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee7ee-3ee4-4c83-a4a5-f885c998dd0b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:34.227693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:34.228325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:34.228601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:34.229372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:34.238273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:34.239702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:34.249043 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1197/2370] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:05:34.517850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:34.519452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:34.533935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af6620-49a4-4fea-99b3-2e1cdff7aa8e None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:34.544742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:34.545666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:34.545947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:34.546216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] PATH_INFO: `/v3/projects/9e243222065440a89e7af5c5ad8a295f` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:34.547429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e243222065440a89e7af5c5ad8a295f)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:34.549320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:34.597915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e243222065440a89e7af5c5ad8a295f', 'request_id': 'req-e44e345b-1568-433c-8e07-b0a332ce0e49'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.603251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e243222065440a89e7af5c5ad8a295f', 'request_id': 'req-e44e345b-1568-433c-8e07-b0a332ce0e49'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.609954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e243222065440a89e7af5c5ad8a295f', 'request_id': 'req-e44e345b-1568-433c-8e07-b0a332ce0e49'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.630413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] The token cache is being invalidate because project 9e243222065440a89e7af5c5ad8a295f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:34.630950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.632504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e44e345b-1568-433c-8e07-b0a332ce0e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:34.635853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1174/2371] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:34 2022] DELETE /identity/v3/projects/9e243222065440a89e7af5c5ad8a295f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:39.369050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:39.370476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:39.383071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f774-e82c-463a-a18a-e9148b8fce24 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:39.436072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:39.436865 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:39.437148 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:39.437425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] PATH_INFO: `/v3/users/400559e7c47e4c0e94b2a539fcbd515e` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:39.438648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=400559e7c47e4c0e94b2a539fcbd515e)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:39.440647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:39.496724 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400559e7c47e4c0e94b2a539fcbd515e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:39.497422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400559e7c47e4c0e94b2a539fcbd515e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:39.504282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400559e7c47e4c0e94b2a539fcbd515e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:39.504943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400559e7c47e4c0e94b2a539fcbd515e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:39.514213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400559e7c47e4c0e94b2a539fcbd515e'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:39.532095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1198/2372] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:39 2022] DELETE /identity/v3/users/400559e7c47e4c0e94b2a539fcbd515e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:39.537281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:39.537824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:39.538122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:39.558540 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:39.560231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:39.654851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1175/2373] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:05:39.664666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:39.666512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:39.736112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d404687-da5d-418e-b7c0-5dab35bd96d8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:39.739844 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:39.740501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:39.741282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:39.741452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:39.760687 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:39.771717 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:39.787370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1199/2374] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:40.080130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:40.081566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:40.095067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54090e46-6418-4e10-b0e9-c732f7f82826 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:40.100102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:40.100975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:40.101258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:40.101668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] PATH_INFO: `/v3/projects/8e1c094fcd8a478db9b0685e7db8f0db` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:40.102965 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1c094fcd8a478db9b0685e7db8f0db)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:40.105018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:40.153477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1c094fcd8a478db9b0685e7db8f0db', 'request_id': 'req-555324f3-2e11-4d73-a8a3-86e9b20b2d54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:40.158666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1c094fcd8a478db9b0685e7db8f0db', 'request_id': 'req-555324f3-2e11-4d73-a8a3-86e9b20b2d54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:40.164652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1c094fcd8a478db9b0685e7db8f0db', 'request_id': 'req-555324f3-2e11-4d73-a8a3-86e9b20b2d54'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:40.176139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] The token cache is being invalidate because project 8e1c094fcd8a478db9b0685e7db8f0db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:40.176427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:40.177101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-555324f3-2e11-4d73-a8a3-86e9b20b2d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:40.179204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1176/2375] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:40 2022] DELETE /identity/v3/projects/8e1c094fcd8a478db9b0685e7db8f0db => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:53.656368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:53.657771 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:53.670170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cb13-9424-427c-97ea-4728d4e1c427 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:53.726265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:53.727157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:53.727439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:53.727712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] PATH_INFO: `/v3/users/4b895c0c03ae4b398ca24924f3f27dd3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:53.728893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b895c0c03ae4b398ca24924f3f27dd3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:53.730850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:53.772057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b895c0c03ae4b398ca24924f3f27dd3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:53.772792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b895c0c03ae4b398ca24924f3f27dd3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:53.779745 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b895c0c03ae4b398ca24924f3f27dd3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:53.780329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b895c0c03ae4b398ca24924f3f27dd3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:53.787206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b895c0c03ae4b398ca24924f3f27dd3'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:53.806351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1200/2376] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:05:53 2022] DELETE /identity/v3/users/4b895c0c03ae4b398ca24924f3f27dd3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:53.810894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:53.811362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:53.811593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:53.830683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:53.832265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:53.913311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1177/2377] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:53.916657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:53.918761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:53.974552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c1fa-2b69-4952-96d6-9e4fed048f89 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:53.978629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:53.979268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:53.979547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:53.979819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:53.988690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:53.990104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462839d-0ba7-4a48-9642-879bd6cd8bc2 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.001753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1201/2378] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:54.260386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.261907 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:54.278570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee19ca-4945-42c5-919f-ac6aba1b52e3 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.283391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.295072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.295360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.295637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] PATH_INFO: `/v3/projects/2ba79aa5a25b4cb98a2c5ce25e1b783d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.299462 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba79aa5a25b4cb98a2c5ce25e1b783d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.301578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.361641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba79aa5a25b4cb98a2c5ce25e1b783d', 'request_id': 'req-74264900-67e0-46b9-bc25-7dc77803517c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:54.368935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba79aa5a25b4cb98a2c5ce25e1b783d', 'request_id': 'req-74264900-67e0-46b9-bc25-7dc77803517c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:54.379641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba79aa5a25b4cb98a2c5ce25e1b783d', 'request_id': 'req-74264900-67e0-46b9-bc25-7dc77803517c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:54.393756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] The token cache is being invalidate because project 2ba79aa5a25b4cb98a2c5ce25e1b783d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:05:54.394257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:54.395192 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-74264900-67e0-46b9-bc25-7dc77803517c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:05:54.398437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1178/2379] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:05:54 2022] DELETE /identity/v3/projects/2ba79aa5a25b4cb98a2c5ce25e1b783d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:54.402079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.402423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.402818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.421822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:54.423082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:54.484768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1202/2380] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:54.492042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.492331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.492605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.511146 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:54.512332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:54.520080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1179/2381] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:54.524721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.534166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab8ed-a993-458d-88a5-b8bfac7b267e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.539002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.539649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.539935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.540210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.540930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:05:54.541236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.542914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.554127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1203/2382] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:05:54.558148 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.567117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0169-1b59-4b73-a9ec-99fd48486d13 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.572199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.572835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.573115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.573388 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.574074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.575995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.599830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1180/2383] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:05:54 2022] POST /identity/v3/projects => generated 363 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:05:54.603791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.613409 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314c2bd-2267-4c5f-92a9-c2cf4110241d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.670195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.670898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.671093 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.671372 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.672063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.673979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.696172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1204/2384] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:05:54 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:05:54.700714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.714263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a4cb-1e37-4186-bed3-e2fd83ec6c4d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.719951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.720684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.720954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.721239 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.721969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.723604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.736189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1181/2385] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:05:54.739817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.750023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2682-1ffb-4b4d-8c2d-04dd8e631542 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.754622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.755277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.755562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.755842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.757081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.758702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd687f-6ebe-471c-b8ce-d38443bb9cac admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.768298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1205/2386] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:05:54.772804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.782701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24469421-2b14-4449-bcc3-27a09a6e1de3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.787441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.788338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.788617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.788888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] PATH_INFO: `/v3/projects/4fe30a3eb82e4bda85fb5c71288bcfe9/users/807cc50431a54d4eb2965ec5ceab5bb6/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.802643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe30a3eb82e4bda85fb5c71288bcfe9, user_id=807cc50431a54d4eb2965ec5ceab5bb6, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:54.805305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:54.817503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1182/2387] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:05:54 2022] PUT /identity/v3/projects/4fe30a3eb82e4bda85fb5c71288bcfe9/users/807cc50431a54d4eb2965ec5ceab5bb6/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:05:54.827976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dabbd-caa4-45fc-b3db-88a826921117 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.828269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dabbd-caa4-45fc-b3db-88a826921117 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.828577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dabbd-caa4-45fc-b3db-88a826921117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.854518 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-722dabbd-caa4-45fc-b3db-88a826921117 None None] MFA Rules not processed for user `807cc50431a54d4eb2965ec5ceab5bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:54.855704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-722dabbd-caa4-45fc-b3db-88a826921117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:54.917788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1206/2388] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:54.929387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:54.931190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:54.985316 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3553d4-4067-431c-a20c-4a6d82cf3a5b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:05:54.989184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:05:54.989853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:54.990137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:54.990441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:54.999845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:05:55.001262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:05:55.010784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1183/2389] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:05:57.152422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c078-1a47-4865-8ad2-d31c1aa13b32 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:05:57.152931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c078-1a47-4865-8ad2-d31c1aa13b32 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:05:57.153026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c078-1a47-4865-8ad2-d31c1aa13b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:05:57.170720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5cc5c078-1a47-4865-8ad2-d31c1aa13b32 None None] MFA Rules not processed for user `06701ca6a5e0427c9d5b540133c522a0`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:05:57.171909 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5cc5c078-1a47-4865-8ad2-d31c1aa13b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:05:57.223559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1207/2390] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:05:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:05:59.969043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:05:59.976075 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:00.089459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c48f3-288e-455b-bd11-ff64d29e2883 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:00.102056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:00.102775 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:00.103058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:00.103335 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:00.109235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:00.125364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:00.127357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:00.142443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1184/2391] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:05:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:06:00.844320 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12469-2ae5-48ae-bb22-c51a485b2515 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:00.844872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12469-2ae5-48ae-bb22-c51a485b2515 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:00.844872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12469-2ae5-48ae-bb22-c51a485b2515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:00.892066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-33a12469-2ae5-48ae-bb22-c51a485b2515 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:00.893263 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-33a12469-2ae5-48ae-bb22-c51a485b2515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:00.954843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1208/2392] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:00.968391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:00.970251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:01.080591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c03157-a374-4dbe-84da-36745e6b0b2c service placement] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:01.089357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:01.090941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:01.090941 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:01.091153 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:01.104627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:01.106767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:01.120459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1185/2393] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:06:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:06:04.484796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff74e5-4bc1-41d2-89d7-ce5cafb64855 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:04.485333 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff74e5-4bc1-41d2-89d7-ce5cafb64855 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:04.485530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff74e5-4bc1-41d2-89d7-ce5cafb64855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:04.652109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d0ff74e5-4bc1-41d2-89d7-ce5cafb64855 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:04.659481 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d0ff74e5-4bc1-41d2-89d7-ce5cafb64855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:04.695603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1209/2394] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:04.741764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:04.768454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0ab0f-c795-472b-96e4-d795ec52bea6 service swift] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:04.789780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:04.790718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:04.791029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:04.791400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:04.812496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:04.814241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8722e-0487-43e7-acf0-6fbe2bee26c8 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:04.840658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1186/2395] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:06:16.341780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75482-88a7-4563-be5d-bab12f451811 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:16.342419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75482-88a7-4563-be5d-bab12f451811 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:16.343115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75482-88a7-4563-be5d-bab12f451811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:16.377800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-d8d75482-88a7-4563-be5d-bab12f451811 None None] MFA Rules not processed for user `e690e810cdb54431af33651d13272239`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:16.378428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d8d75482-88a7-4563-be5d-bab12f451811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:16.464827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1210/2396] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:25.329149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:25.330044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:25.330558 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:25.334066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1187/2397] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:06:25.341727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382611-f677-4f1a-be02-17b11526b149 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:25.342982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382611-f677-4f1a-be02-17b11526b149 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:25.343415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382611-f677-4f1a-be02-17b11526b149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:25.409544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0a382611-f677-4f1a-be02-17b11526b149 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:25.410787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0a382611-f677-4f1a-be02-17b11526b149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:25.493436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1211/2398] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:26.479459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:26.481241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:26.576480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5c18-e1f9-4998-ab2c-d274fbe96f01 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:26.580091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:26.580737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.581019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.581918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:26.592721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:26.594113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042a0e5-1621-4792-a9aa-6b911c44ae21 service glance] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:26.606598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1188/2399] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:06:26.727015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.727622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.727834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:26.729830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1212/2400] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:06:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:06:26.735649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3879-c229-4a56-b3d5-59258640a94f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.736106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3879-c229-4a56-b3d5-59258640a94f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.736553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3879-c229-4a56-b3d5-59258640a94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:26.762377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b01e3879-c229-4a56-b3d5-59258640a94f None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:26.764840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b01e3879-c229-4a56-b3d5-59258640a94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:26.829959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1189/2401] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:26.842197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:26.843961 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:26.863227 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b46cb-5843-4730-80a2-9a2f899c9f95 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:26.866860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:26.867527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.867839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.868114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:26.875929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:26.877323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c047ec7-c092-4ab3-9b1b-1779e174726b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:26.885438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1213/2402] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:06:26.980067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.980442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.980722 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:26.982130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1190/2403] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:06:26.987620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ce20-45a9-47d4-a228-af9bace7a26d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:26.987908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ce20-45a9-47d4-a228-af9bace7a26d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:26.988186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ce20-45a9-47d4-a228-af9bace7a26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:27.007105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9ce3ce20-45a9-47d4-a228-af9bace7a26d None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:27.008363 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9ce3ce20-45a9-47d4-a228-af9bace7a26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:27.017810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1214/2404] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:27.028711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:27.030490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:27.043933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515e8c2-aee4-4246-b29c-de024e2bafdd None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:27.047519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:27.048157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:27.048439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:27.048717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:27.058912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:27.060515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:27.071323 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1191/2405] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:06:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:06:27.598893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:27.599185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:27.599483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:27.617156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:27.618459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:27.625875 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1215/2406] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:27.638817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:27.649599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18d260-d26f-4c7f-ae89-d0d7c5204246 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:27.654733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:27.655299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:27.655577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:27.655848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:27.664224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:27.665795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:27.673845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1192/2407] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:06:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:06:29.666733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:29.668311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:29.740622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc55d-baa9-47a9-833a-1ce3d39196bd None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:29.747304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:29.748228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:29.748535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:29.748824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] PATH_INFO: `/v3/users/3bd41b34f5b14ae0a0ebccec9d59944f` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:29.750284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd41b34f5b14ae0a0ebccec9d59944f)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:29.753532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:29.811353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd41b34f5b14ae0a0ebccec9d59944f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:29.812305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd41b34f5b14ae0a0ebccec9d59944f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:29.822198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd41b34f5b14ae0a0ebccec9d59944f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:29.822927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd41b34f5b14ae0a0ebccec9d59944f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:29.830739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9ffc2915-f8dd-42d3-b5fc-4355c8ec6420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd41b34f5b14ae0a0ebccec9d59944f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:29.854572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1216/2408] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:06:29 2022] DELETE /identity/v3/users/3bd41b34f5b14ae0a0ebccec9d59944f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:06:30.438613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:30.453813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb238ae-9ab4-444a-a0da-1c0bfac1e5b1 service swift] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:30.524653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:30.525225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:30.525503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:30.525773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] PATH_INFO: `/v3/projects/c9d3d605483a4228bd54793a27723c91` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:30.526987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d3d605483a4228bd54793a27723c91)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:30.528873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:30.581243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d3d605483a4228bd54793a27723c91', 'request_id': 'req-d562308e-e64b-4a7c-b7a9-f8fad2abc248'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:30.587080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d3d605483a4228bd54793a27723c91', 'request_id': 'req-d562308e-e64b-4a7c-b7a9-f8fad2abc248'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:30.594102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d3d605483a4228bd54793a27723c91', 'request_id': 'req-d562308e-e64b-4a7c-b7a9-f8fad2abc248'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:30.608213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] The token cache is being invalidate because project c9d3d605483a4228bd54793a27723c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:06:30.608499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:30.609117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:30.610803 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1193/2409] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:06:30 2022] DELETE /identity/v3/projects/c9d3d605483a4228bd54793a27723c91 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:06:31.152190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:31.152731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:31.152829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:31.170411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:31.171615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:31.229633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1217/2410] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:31.242536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:31.243921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:31.313242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562308e-e64b-4a7c-b7a9-f8fad2abc248 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:31.319603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:31.320268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:31.320550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:31.320824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:31.329413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:31.331042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fde3c5-45e2-4160-b992-b63c8a359dd8 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:31.339395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1194/2411] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:06:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:06:47.161746 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:47.163848 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.230598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6babcd-0e0b-4ded-a743-462bc4478287 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:47.235962 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:47.236836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.237122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.237395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] PATH_INFO: `/v3/users/807cc50431a54d4eb2965ec5ceab5bb6` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.238734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=807cc50431a54d4eb2965ec5ceab5bb6)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:47.240779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:47.288083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807cc50431a54d4eb2965ec5ceab5bb6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.289046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807cc50431a54d4eb2965ec5ceab5bb6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.295674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807cc50431a54d4eb2965ec5ceab5bb6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.296294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807cc50431a54d4eb2965ec5ceab5bb6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.303377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807cc50431a54d4eb2965ec5ceab5bb6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.321388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1218/2412] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:06:47 2022] DELETE /identity/v3/users/807cc50431a54d4eb2965ec5ceab5bb6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:06:47.325831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.326427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.326427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.344638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:47.345829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.403381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1195/2413] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:47.413087 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:47.414523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.471935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbad5c-2bfc-4b75-bcf1-98f4e10d5b3d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:47.475615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:47.476291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.476572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.476847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.485888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:47.487335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd6c9-cd25-4734-91e6-9fcd4efa4dc8 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:47.497099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1219/2414] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:06:47.749897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:47.751431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.763812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62099a87-ebe6-4850-9371-0be0e253f033 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:47.768546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:47.769905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.770226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.770495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] PATH_INFO: `/v3/projects/4fe30a3eb82e4bda85fb5c71288bcfe9` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.771771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe30a3eb82e4bda85fb5c71288bcfe9)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:47.774836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:47.820489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe30a3eb82e4bda85fb5c71288bcfe9', 'request_id': 'req-f810caf5-f3d1-436c-b707-8f7097dfded9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.829809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe30a3eb82e4bda85fb5c71288bcfe9', 'request_id': 'req-f810caf5-f3d1-436c-b707-8f7097dfded9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.836653 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe30a3eb82e4bda85fb5c71288bcfe9', 'request_id': 'req-f810caf5-f3d1-436c-b707-8f7097dfded9'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.850496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] The token cache is being invalidate because project 4fe30a3eb82e4bda85fb5c71288bcfe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:06:47.850861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.851594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-f810caf5-f3d1-436c-b707-8f7097dfded9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:06:47.853038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1196/2415] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:06:47 2022] DELETE /identity/v3/projects/4fe30a3eb82e4bda85fb5c71288bcfe9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:06:47.857872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.858151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.858526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.876556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:47.877775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.938488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1220/2416] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:06:47.946680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.946974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.947257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.965904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:47.967486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:47.975264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1197/2417] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:06:47.979461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:47.988889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272203de-c437-4c3e-beb9-91b68630fbf1 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:47.994078 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:47.994770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:47.995059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:47.995337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:47.996062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:06:47.996369 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:47.998243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.010676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1221/2418] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:06:48.014830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.024138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681667d9-d827-4b5e-a4c3-cd91b6392088 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.029139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.029794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.030078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.030353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.031082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.032974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.058365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1198/2419] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:06:48 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:06:48.062634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.073828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d244-d06e-4140-9604-5128007d8fcc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.131152 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.131825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.132690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.132974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.133663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.135604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.155579 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1222/2420] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:06:48 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:06:48.159493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.170860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ab74-41d6-407d-99f8-d41e6e49e651 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.176330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.177107 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.177490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.177842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.178925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.182049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.197754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1199/2421] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:06:48.199269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.213022 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef3cb-1391-49f5-90cf-e29386b1b61a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.217937 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.218674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.218951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.219226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.219945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.221587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582934e9-360a-497a-8dbd-26793442b45e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.234012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1223/2422] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:06:48.238140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.250468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a9ce-b451-435d-aced-01084193c3ec admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.255730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.256620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.256927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.257213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] PATH_INFO: `/v3/projects/be8a0fa7a64d497eb4ea406390755f9b/users/d1507b4dfa1a460eaf1dcfd35e459f73/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.275233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8a0fa7a64d497eb4ea406390755f9b, user_id=d1507b4dfa1a460eaf1dcfd35e459f73, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.278098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.295741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1200/2423] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:06:48 2022] PUT /identity/v3/projects/be8a0fa7a64d497eb4ea406390755f9b/users/d1507b4dfa1a460eaf1dcfd35e459f73/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:06:48.306255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edba3-1615-4521-9e44-9d56ed0fa187 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.306586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edba3-1615-4521-9e44-9d56ed0fa187 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.306870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edba3-1615-4521-9e44-9d56ed0fa187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.332382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0b3edba3-1615-4521-9e44-9d56ed0fa187 None None] MFA Rules not processed for user `d1507b4dfa1a460eaf1dcfd35e459f73`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:06:48.333566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0b3edba3-1615-4521-9e44-9d56ed0fa187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:48.398101 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1224/2424] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:06:48.408961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:48.410727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:48.466690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bba492-39e5-4e91-9a38-48075842c7cb admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:48.470290 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:48.470974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:48.471252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:48.471519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:48.480233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:48.481662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:48.490642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1201/2425] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:06:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:07:06.613242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:06.613781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:06.613781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:06.631889 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:07:06.633132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:06.687384 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1225/2426] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:07:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:07:06.700188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:06.701567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:06.768318 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe6d3f-ce82-41a6-8625-2f7d4ff39c44 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:06.775463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:06.776100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:06.776385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:06.776654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:06.785475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:06.787291 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ce2dd-a55b-4c12-953f-771a5e87f1e9 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:06.796180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1202/2427] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:07:08.749032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:08.808815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926e2db-b1d6-4c87-90b1-87e37d80617e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:08.813258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:08.814121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:08.814424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:08.814707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] PATH_INFO: `/v3/users/d1507b4dfa1a460eaf1dcfd35e459f73` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:08.815895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1507b4dfa1a460eaf1dcfd35e459f73)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:08.817806 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:08.859738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1507b4dfa1a460eaf1dcfd35e459f73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:08.860403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1507b4dfa1a460eaf1dcfd35e459f73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:08.870371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1507b4dfa1a460eaf1dcfd35e459f73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:08.871019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1507b4dfa1a460eaf1dcfd35e459f73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:08.878238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1507b4dfa1a460eaf1dcfd35e459f73'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:08.895439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1226/2428] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:07:08 2022] DELETE /identity/v3/users/d1507b4dfa1a460eaf1dcfd35e459f73 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:07:08.899883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:08.900391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:08.900490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:08.917565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:07:08.918792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:08.973159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1203/2429] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:07:08.982721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:08.984029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:09.038969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b04f07-5937-4c34-84f3-e5b152366d12 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:09.042533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:09.043160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:09.043443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:09.043716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:09.052329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:09.053723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476c28-8dc5-4fb1-a09e-a080704c4760 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:09.063444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1227/2430] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:07:09.313542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:09.315029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:09.327656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad19d95-5937-4604-a8e0-a073eb5235af None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:09.332200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:09.333046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:09.333325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:09.333598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] PATH_INFO: `/v3/projects/be8a0fa7a64d497eb4ea406390755f9b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:09.334848 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8a0fa7a64d497eb4ea406390755f9b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:09.337435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:09.381101 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8a0fa7a64d497eb4ea406390755f9b', 'request_id': 'req-c1bb8dc6-d9b4-434a-a578-be80b89537d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:09.387276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8a0fa7a64d497eb4ea406390755f9b', 'request_id': 'req-c1bb8dc6-d9b4-434a-a578-be80b89537d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:09.393301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8a0fa7a64d497eb4ea406390755f9b', 'request_id': 'req-c1bb8dc6-d9b4-434a-a578-be80b89537d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:09.405683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] The token cache is being invalidate because project be8a0fa7a64d497eb4ea406390755f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:07:09.405966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:09.406606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:07:09.408166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1204/2431] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:07:09 2022] DELETE /identity/v3/projects/be8a0fa7a64d497eb4ea406390755f9b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:07:43.855897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:43.856410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:43.856526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:43.874066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] MFA Rules not processed for user `06701ca6a5e0427c9d5b540133c522a0`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:07:43.875272 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:43.927252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1228/2432] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:07:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:07:53.856606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:53.858086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:53.913026 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8dc6-d9b4-434a-a578-be80b89537d8 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:53.917348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:53.917998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:53.918287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:53.918598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:53.920444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:53.989248 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:53.990720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:54.002010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1205/2433] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:07:56.210408 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:07:56.212719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:56.267419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c4fa7-b012-4507-a17d-d49bf49135d0 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:07:56.271047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:07:56.271693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:07:56.271975 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:07:56.272250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:07:56.274770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:07:56.289005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:07:56.290563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa0abf-25d7-447e-b7ab-9f7450572c49 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:07:56.302576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1229/2434] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:02.303828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:02.305687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:02.319081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fa3b5-4732-4887-888d-bd159ba8c6ec service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:02.325574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:02.326219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:02.326534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:02.326918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:02.328797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:02.386282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:02.387945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:02.397301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1206/2435] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:08.313163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:08.314079 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:08.314355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:08.332888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] MFA Rules not processed for user `aa56c83ab8494901b85d00d4f592e5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:08:08.334556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:08.391944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1230/2436] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:08:08.682278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:08.683701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:08.694790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bb6c8-fe83-448e-b706-b94007b9deb6 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:08.698903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:08.699526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:08.699804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:08.700073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:08.701714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:08.710591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:08.711986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:08.719602 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1207/2437] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:09.018935 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:09.020336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:09.089398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0b9d5-3599-4777-bf29-2db1f8f3d204 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:09.094276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:09.094938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:09.095220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:09.095492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:09.097218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:09.108930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:09.110535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:09.120739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1231/2438] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:10.900822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:10.910011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032bce-1aaa-4271-a947-cc2965b296c0 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:10.914594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:10.915308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:10.915681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:10.915950 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:10.917883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:10.928651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:10.930662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255e500-fcc9-43eb-b884-a1d5f0399359 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:10.938325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1208/2439] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:19.805074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:19.806625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:19.822513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b842e50-6037-419f-8946-60e9d20caf56 service placement] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:19.827915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:19.828570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:19.828846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:19.829120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:19.830898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:19.843569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:19.845178 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:19.855553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1232/2440] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:22.514634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5df18-14e8-4c56-ad15-903da91ef91c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:22.515150 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5df18-14e8-4c56-ad15-903da91ef91c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:22.515247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5df18-14e8-4c56-ad15-903da91ef91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:22.532409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-d6f5df18-14e8-4c56-ad15-903da91ef91c None None] MFA Rules not processed for user `e690e810cdb54431af33651d13272239`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:08:22.533590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d6f5df18-14e8-4c56-ad15-903da91ef91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:22.541043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1209/2441] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:08:39.996405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:08:40.007483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10485a9a-9581-4418-9791-339c10982ce6 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:08:40.012813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:08:40.013438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:40.013720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:40.013995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:40.015753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:40.025648 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:08:40.027315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:08:40.035952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1233/2442] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:08:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:08:57.083793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238571c2-812e-4a39-a210-458d811ee568 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:08:57.084295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238571c2-812e-4a39-a210-458d811ee568 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:08:57.084392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238571c2-812e-4a39-a210-458d811ee568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:08:57.102870 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-238571c2-812e-4a39-a210-458d811ee568 None None] MFA Rules not processed for user `06701ca6a5e0427c9d5b540133c522a0`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:08:57.104053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-238571c2-812e-4a39-a210-458d811ee568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:08:57.110928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1210/2443] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:04.205313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:04.216105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e51dbb-a0be-4167-bc6a-c0b147e96972 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:04.221447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:04.222069 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:04.222353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:04.222661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:04.224355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:04.234251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:04.235847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:04.245112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1234/2444] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:04.657086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238571c2-812e-4a39-a210-458d811ee568 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:04.670304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238571c2-812e-4a39-a210-458d811ee568 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:04.675966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:04.676620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:04.676900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:04.677170 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:04.678984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:04.692063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:04.693772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:04.704900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1211/2445] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:08.251547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:08.263061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180b00-d740-4a7c-b7b3-eca81bd3eca6 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:08.268461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:08.269089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:08.269367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:08.269639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:08.271499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:08.283447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:08.285014 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:08.295839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1235/2446] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:26.220182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:26.229738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ea342-0866-4ef8-a5fe-8ae4b6e7d80a service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:26.233499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:26.234120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:26.234411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:26.234689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:26.236395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:26.296580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:26.297995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:26.307246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1212/2447] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:29.729362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:29.729362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:29.738083 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f7fb-6135-4cdb-998a-50123f1e6e2e service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:29.742700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:29.744090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:29.744373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:29.744688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] PATH_INFO: `/v3/users/e690e810cdb54431af33651d13272239` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:29.745828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e690e810cdb54431af33651d13272239)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:29.748053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:29.792929 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e690e810cdb54431af33651d13272239'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:29.793502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e690e810cdb54431af33651d13272239'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:29.800015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e690e810cdb54431af33651d13272239'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:29.800641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e690e810cdb54431af33651d13272239'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:29.807873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71fc7034-34d9-4a1c-9f85-bcbf337fdcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e690e810cdb54431af33651d13272239'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:29.825084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1236/2448] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:29 2022] DELETE /identity/v3/users/e690e810cdb54431af33651d13272239 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:29.994833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.006071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b17681-da2d-472b-a403-6a4a34d7d5ec service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.063166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.064023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.064301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.064572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] PATH_INFO: `/v3/projects/d851fe67b7e940f9b20979bdd619bf48` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.065850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d851fe67b7e940f9b20979bdd619bf48)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.068545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.113649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd851fe67b7e940f9b20979bdd619bf48', 'request_id': 'req-7642dcf2-7056-4890-837e-8fd79b55a611'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:30.119139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd851fe67b7e940f9b20979bdd619bf48', 'request_id': 'req-7642dcf2-7056-4890-837e-8fd79b55a611'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:30.125445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd851fe67b7e940f9b20979bdd619bf48', 'request_id': 'req-7642dcf2-7056-4890-837e-8fd79b55a611'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:30.138680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] The token cache is being invalidate because project d851fe67b7e940f9b20979bdd619bf48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:30.138966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:30.139607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7642dcf2-7056-4890-837e-8fd79b55a611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:30.141304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1213/2449] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:29 2022] DELETE /identity/v3/projects/d851fe67b7e940f9b20979bdd619bf48 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:30.146630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.146925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.147203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.164595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:30.165812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.237215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1237/2450] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:30.238820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf4cfc-76ec-4217-8a28-6b457bad34cb None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.239142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf4cfc-76ec-4217-8a28-6b457bad34cb None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.239521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf4cfc-76ec-4217-8a28-6b457bad34cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.259468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-ffbf4cfc-76ec-4217-8a28-6b457bad34cb None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:30.260681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ffbf4cfc-76ec-4217-8a28-6b457bad34cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.272155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1214/2451] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:30.274565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.287774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50ceb8-f69b-47ee-87d9-b5ecc7c45a15 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.292928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.294314 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.294624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.294897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.296124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:30.296277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.297873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.307555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.307847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.308128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.313403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1238/2452] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:09:30.314536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.324222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3dad9-d879-40d7-b706-ab9219a11a2e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.325809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:30.327047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.329027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.329693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.329972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.330279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.330985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.332864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.371494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1239/2453] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:09:30.372740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.383714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1215/2454] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:30.437475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.438920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.445407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea9c7-7a8c-4804-8704-3fc7b15ce0ab admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.451144 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.451822 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.452105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.452378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.453096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.455062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.478117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1240/2455] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:30.479279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.489479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debb4c9-dfec-4280-851a-cd7b30c9de1c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.495955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.497122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.497610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.497919 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.498638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.500236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.511055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1241/2456] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:30.514979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.516911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d27b86-504c-48d8-a985-d0de2b8f7000 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.525008 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b00a-2857-407d-947d-92a4708bab0f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.531054 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.531725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.532012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.532289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.533013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.534632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.550633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1242/2457] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:09:30.552931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.563943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5fdad-6c0c-41c7-8fc7-2df570199400 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.568685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.569576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.569856 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.570129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] PATH_INFO: `/v3/projects/13903400ae4744c6aab71bda83e84118/users/104c29b17a454c95ae38fd37c692e1de/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.584483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.585727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.586009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.586235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13903400ae4744c6aab71bda83e84118, user_id=104c29b17a454c95ae38fd37c692e1de, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.586371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.590313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4372348-8ba5-430a-bd5b-29ddc56e5686 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.607749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1243/2458] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:30 2022] PUT /identity/v3/projects/13903400ae4744c6aab71bda83e84118/users/104c29b17a454c95ae38fd37c692e1de/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:30.618760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.619567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.619883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.647758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] MFA Rules not processed for user `104c29b17a454c95ae38fd37c692e1de`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:30.648974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.649660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.651281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:30.659954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1216/2459] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 15:09:30.749039 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1244/2460] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:30.761093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:30.762644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:30.865712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139105f-8445-406d-b1c2-444d827eba25 service swift] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:30.964422 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:30.965413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:30.965413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:30.965773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:30.975417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:30.977736 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e737177-ef7b-4c05-9dbe-c675b93c3536 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:31.002263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1217/2461] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:31.247264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:31.313156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912bbf5-9ab2-4aba-bedb-224e45fcbf53 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:31.317754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:31.318689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:31.318973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:31.319248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] PATH_INFO: `/v3/users/104c29b17a454c95ae38fd37c692e1de` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:31.320447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104c29b17a454c95ae38fd37c692e1de)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:31.322420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:31.373460 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104c29b17a454c95ae38fd37c692e1de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:31.374302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104c29b17a454c95ae38fd37c692e1de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:31.381921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104c29b17a454c95ae38fd37c692e1de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:31.382725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104c29b17a454c95ae38fd37c692e1de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:31.390770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104c29b17a454c95ae38fd37c692e1de'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:31.410179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1245/2462] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:31 2022] DELETE /identity/v3/users/104c29b17a454c95ae38fd37c692e1de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:31.417261 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:31.417940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:31.418496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:31.437012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:31.438499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:31.502098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1218/2463] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:31.512863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:31.514256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:31.576154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863f967-3073-4e69-b7a3-376577836057 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:31.580232 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:31.581131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:31.581557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:31.582055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:31.594859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:31.596294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751bffb-477b-4406-81b5-8705381d9c32 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:31.609799 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1246/2464] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:09:31.919842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:31.919842 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:31.942092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76816d-c0d7-424b-ba20-c4be14c809f9 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:31.948605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:31.949505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:31.950724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:31.951312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] PATH_INFO: `/v3/projects/13903400ae4744c6aab71bda83e84118` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:31.953086 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13903400ae4744c6aab71bda83e84118)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:31.955453 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.027846 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13903400ae4744c6aab71bda83e84118', 'request_id': 'req-bc998f84-6828-4cbe-ad90-f6ac06bddda2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:32.035489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13903400ae4744c6aab71bda83e84118', 'request_id': 'req-bc998f84-6828-4cbe-ad90-f6ac06bddda2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:32.044059 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13903400ae4744c6aab71bda83e84118', 'request_id': 'req-bc998f84-6828-4cbe-ad90-f6ac06bddda2'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:32.058767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] The token cache is being invalidate because project 13903400ae4744c6aab71bda83e84118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:32.059301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:32.060095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-bc998f84-6828-4cbe-ad90-f6ac06bddda2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:32.065537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1219/2465] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:31 2022] DELETE /identity/v3/projects/13903400ae4744c6aab71bda83e84118 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:32.067695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.068205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.068308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.103229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:32.104601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:32.176546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1247/2466] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:32.184856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.185145 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.185427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.205167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:32.206411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:32.215818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1220/2467] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:32.219629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.231322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e018282-3978-4cc6-a0c9-8b488e67ff11 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.237071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.237785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.238067 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.238349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.239103 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:32.239411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.241023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.257872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1248/2468] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:09:32.259052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.268570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e500de-8afe-4ad2-89a4-0e23029a2d4d None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.273569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.274529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.274839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.275114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.275824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.277880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.311010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1221/2469] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:09:32.313474 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.324512 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0552dac-c4e9-454a-949b-0d589f6df5af admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.399124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.399809 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.400097 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.400380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.401074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.403060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.426501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1249/2470] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:32.429624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.442883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a30ec-36da-4167-ac53-13b69badfece admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.448015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.449565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.449565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.449565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.450296 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.451901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.464828 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1222/2471] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:32.467655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.479650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30ca5-3f3c-4b14-930b-857f471ccf24 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.484566 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.485246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.485532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.485812 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.486572 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.488170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3aeb2-9337-4771-aaa8-ec1b07204a1a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.500386 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1250/2472] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:32.504366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.514821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69cc9-145a-4986-afb0-4e2ba34ef997 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.520091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.520983 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.521279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.521589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] PATH_INFO: `/v3/projects/743afe2c8f214eebb2ce6312b24002e6/users/380ec2e5d10a4c31816c3828ed821204/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.569692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743afe2c8f214eebb2ce6312b24002e6, user_id=380ec2e5d10a4c31816c3828ed821204, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.572895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.601824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1223/2473] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:32 2022] PUT /identity/v3/projects/743afe2c8f214eebb2ce6312b24002e6/users/380ec2e5d10a4c31816c3828ed821204/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:32.603681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.603967 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.605465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.636741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] MFA Rules not processed for user `380ec2e5d10a4c31816c3828ed821204`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:32.638304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:32.718468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1251/2474] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:32.723000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.824028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435050e-e5d8-402e-aaf2-591ec195ecf2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.829676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.830324 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.830644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.830921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.831763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.833700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.874639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1224/2475] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:32.875829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:32.888855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda7ff-ff66-487e-a758-c550eb1bd7b4 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:32.960643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:32.961243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:32.962262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:32.962591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:32.963292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:32.965224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:32.987176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1252/2476] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:32 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:09:32.990813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.001829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af5e3-667d-4197-9fa1-8903efd84b05 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.008173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.010219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.010711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.011135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.012005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.013906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.025795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1225/2477] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:33.030761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.041186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402b63-76f6-4997-80c6-48cd31a47860 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.046284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.047215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.047549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.047832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] PATH_INFO: `/v3/projects/0919f3daf05347ef8166651957b96f71/users/9ecfd35b95994f0a860870b011f907d8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.063587 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0919f3daf05347ef8166651957b96f71, user_id=9ecfd35b95994f0a860870b011f907d8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.066308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.083563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1253/2478] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:33 2022] PUT /identity/v3/projects/0919f3daf05347ef8166651957b96f71/users/9ecfd35b95994f0a860870b011f907d8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:33.088250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.100353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f56b-dffc-430e-9f63-95480525064d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.180833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.181528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.181812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.182118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.182890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.184532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0854f-aa6e-4903-9803-3c6ba0e5ff00 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.201089 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1226/2479] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:09:33.202368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.218121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e9516-35d8-4c38-a009-19cb85513d4b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.223469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.224430 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.224782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.225259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] PATH_INFO: `/v3/projects/0919f3daf05347ef8166651957b96f71/users/9ecfd35b95994f0a860870b011f907d8/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.227561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0919f3daf05347ef8166651957b96f71, user_id=9ecfd35b95994f0a860870b011f907d8, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.243607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.257533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1254/2480] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:33 2022] PUT /identity/v3/projects/0919f3daf05347ef8166651957b96f71/users/9ecfd35b95994f0a860870b011f907d8/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:33.270970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.271390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.271867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.302631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] MFA Rules not processed for user `9ecfd35b95994f0a860870b011f907d8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:33.304102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:33.389368 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1227/2481] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:33.400624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.402605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:33.467441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ce4b-f8cf-40b0-9a3d-d73911f3ee0b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.472282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.472992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.473357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.473698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.484084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.485757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b759b8-ddfe-4152-9a26-3a0df3b89e71 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.499202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1255/2482] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:33.569477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.637547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba6cf-df1c-4b82-969a-2c078ca007b7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.642215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.643200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.646611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.646611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] PATH_INFO: `/v3/users/380ec2e5d10a4c31816c3828ed821204` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.646611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=380ec2e5d10a4c31816c3828ed821204)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.648027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:33.719066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '380ec2e5d10a4c31816c3828ed821204'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:33.724794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '380ec2e5d10a4c31816c3828ed821204'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:33.725488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '380ec2e5d10a4c31816c3828ed821204'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:33.726088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '380ec2e5d10a4c31816c3828ed821204'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:33.755579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '380ec2e5d10a4c31816c3828ed821204'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:33.776030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1228/2483] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:33 2022] DELETE /identity/v3/users/380ec2e5d10a4c31816c3828ed821204 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:33.780558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.780847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.781124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.801651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:33.802861 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:33.877807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1256/2484] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:33.885683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:33.888120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:33.973958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fbfe5-fc41-4086-872b-3a6856ae0977 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:33.978288 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:33.979030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:33.979320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:33.979593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:33.991262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:33.993811 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:34.005934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1229/2485] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:09:34.308639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:34.310059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:34.322944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7db28-79e7-49ce-b45f-a82435ccbb7e None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:34.329938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:34.331459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:34.332843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:34.333153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] PATH_INFO: `/v3/projects/743afe2c8f214eebb2ce6312b24002e6` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:34.335912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743afe2c8f214eebb2ce6312b24002e6)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:34.341065 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:34.399908 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743afe2c8f214eebb2ce6312b24002e6', 'request_id': 'req-71de2b1e-704a-43be-994c-37ffb3f10a08'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.411943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743afe2c8f214eebb2ce6312b24002e6', 'request_id': 'req-71de2b1e-704a-43be-994c-37ffb3f10a08'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.418693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743afe2c8f214eebb2ce6312b24002e6', 'request_id': 'req-71de2b1e-704a-43be-994c-37ffb3f10a08'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.445734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] The token cache is being invalidate because project 743afe2c8f214eebb2ce6312b24002e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:34.446091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.446743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.448527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1257/2486] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:34 2022] DELETE /identity/v3/projects/743afe2c8f214eebb2ce6312b24002e6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:34.452545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:34.464839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1fc78-5579-48e9-bd37-0d88df400a7c service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:34.529247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:34.530278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:34.530577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:34.530854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] PATH_INFO: `/v3/users/9ecfd35b95994f0a860870b011f907d8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:34.532500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ecfd35b95994f0a860870b011f907d8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:34.534869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:34.594521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ecfd35b95994f0a860870b011f907d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.603711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ecfd35b95994f0a860870b011f907d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.604826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ecfd35b95994f0a860870b011f907d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.605467 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ecfd35b95994f0a860870b011f907d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.619956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-e0eaaab1-2dd8-409d-a85b-5840943cb068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ecfd35b95994f0a860870b011f907d8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:34.653366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1230/2487] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:34 2022] DELETE /identity/v3/users/9ecfd35b95994f0a860870b011f907d8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:34.973258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:34.974900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:34.990015 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de2b1e-704a-43be-994c-37ffb3f10a08 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.056930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.058019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.058299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.058638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] PATH_INFO: `/v3/projects/0919f3daf05347ef8166651957b96f71` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.059995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0919f3daf05347ef8166651957b96f71)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.061926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.112786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0919f3daf05347ef8166651957b96f71', 'request_id': 'req-e9ce5732-f786-4ccd-9c78-adc4369b284a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:35.121146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0919f3daf05347ef8166651957b96f71', 'request_id': 'req-e9ce5732-f786-4ccd-9c78-adc4369b284a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:35.127001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0919f3daf05347ef8166651957b96f71', 'request_id': 'req-e9ce5732-f786-4ccd-9c78-adc4369b284a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:35.140555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] The token cache is being invalidate because project 0919f3daf05347ef8166651957b96f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:35.140867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:35.141482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e9ce5732-f786-4ccd-9c78-adc4369b284a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:35.144569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1258/2488] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:34 2022] DELETE /identity/v3/projects/0919f3daf05347ef8166651957b96f71 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:35.149350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.149629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.149909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.170488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:35.171847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:35.235112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1231/2489] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:35.242913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.243212 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.243476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.264712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:35.265893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:35.273983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1259/2490] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:35.278699 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.288546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a9f87-6e8e-4c2e-a7a0-93ac51ab91aa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.294421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.295077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.295356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.295754 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.296490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:35.296882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.299969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.316549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1232/2491] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:09:35.317419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.326952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de49e89-caa1-4cfb-959d-bd21b0cb90d3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.331753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.332494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.332843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.333213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.333922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.335831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.371552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1260/2492] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:09:35.373396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.384544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d388-fe07-49fd-ab30-f213036086a5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.458212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.458923 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.459227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.459505 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.460183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.462935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.483813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1233/2493] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:35.487720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.499633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaeb85e-58e7-44c6-a529-57acc43bcd68 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.504467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.505229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.505584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.505922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.506741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.508496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.522115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1261/2494] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:35.523544 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.534580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c10d-8315-43f1-a5a3-200ba4d1f411 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.540188 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.540854 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.541137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.541406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.542118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.543834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96e98-3453-4cae-b19c-e75eb00d9f2a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.558496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1234/2495] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:35.559268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.569788 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883dbd-242c-443c-ac0b-87d9531413c2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.575041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.576198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.576693 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.576965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] PATH_INFO: `/v3/projects/d45bc9a11cb14db9a3384262b5c9a37b/users/29466df13aa540d085240203e26c74fa/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.593090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45bc9a11cb14db9a3384262b5c9a37b, user_id=29466df13aa540d085240203e26c74fa, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.596018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.612221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1262/2496] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:35 2022] PUT /identity/v3/projects/d45bc9a11cb14db9a3384262b5c9a37b/users/29466df13aa540d085240203e26c74fa/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:35.623187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.623380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.623673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.652997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] MFA Rules not processed for user `29466df13aa540d085240203e26c74fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:35.654245 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:35.741890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1235/2497] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:35.743281 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.825122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372f97-bf2e-4ea5-9dcd-b5883b4a9049 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.832225 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.832918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.833206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.833499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.834632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.837384 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:35.874913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1263/2498] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:35.876474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:35.888365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd737af-4dd4-4edf-83e6-9c14e5ae8179 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:35.962307 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:35.963475 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:35.963975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:35.964425 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:35.965415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:35.967921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.001876 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1236/2499] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:35 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:36.005497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.018847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47f615-0946-40f1-afe9-316fb7595167 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.025990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.027510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.027989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.028439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.029976 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.031774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.046428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1264/2500] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:36.050494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.061269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82237-5f8c-4421-8c90-c82a3014a095 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.066938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.067825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.068112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.068392 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] PATH_INFO: `/v3/projects/7979695395494dc9a7073952bd6699b4/users/6577735a658344f6a2228aab6d92ded7/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.085096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7979695395494dc9a7073952bd6699b4, user_id=6577735a658344f6a2228aab6d92ded7, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.088402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.104482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1237/2501] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:36 2022] PUT /identity/v3/projects/7979695395494dc9a7073952bd6699b4/users/6577735a658344f6a2228aab6d92ded7/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:36.108480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.119753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf66f5-8e49-42fd-a14b-c300e574e14c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.179964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.181070 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.181358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.181637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.182357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.184443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7716e7-2be6-4c6a-9652-a36e4fd23034 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.198364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1265/2502] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:36.199637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.210404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bda4e-d6fb-471f-a54f-99b70fd38fd0 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.215871 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.217548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.217825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.218105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] PATH_INFO: `/v3/projects/7979695395494dc9a7073952bd6699b4/users/6577735a658344f6a2228aab6d92ded7/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.220127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7979695395494dc9a7073952bd6699b4, user_id=6577735a658344f6a2228aab6d92ded7, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.223102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.234081 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1238/2503] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:36 2022] PUT /identity/v3/projects/7979695395494dc9a7073952bd6699b4/users/6577735a658344f6a2228aab6d92ded7/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:36.245228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.245515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.245789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.275551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] MFA Rules not processed for user `6577735a658344f6a2228aab6d92ded7`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:36.276517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:36.347499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1266/2504] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:36.359684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.361377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:36.423676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3177b-735e-4e21-a375-1caa071d1924 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.427520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.428193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.428471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.428743 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.438972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.440358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.451315 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1239/2505] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:36.580209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:36.592210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffd95-05d3-40d8-98d1-ea3a6df719d2 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:36.595859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:36.596524 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:36.596804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:36.597074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:36.668349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:36.669792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2396aa1-de19-4931-91a6-af08af813025 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:36.682741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1267/2506] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:37.045890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:37.140584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d8554-569a-4c74-b411-1d30e6e78629 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:37.145577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:37.146843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:37.147134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:37.147418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] PATH_INFO: `/v3/users/29466df13aa540d085240203e26c74fa` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:37.148778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=29466df13aa540d085240203e26c74fa)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:37.150812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:37.211384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29466df13aa540d085240203e26c74fa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.220604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29466df13aa540d085240203e26c74fa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.221270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29466df13aa540d085240203e26c74fa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.221886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29466df13aa540d085240203e26c74fa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.230468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29466df13aa540d085240203e26c74fa'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.254420 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1240/2507] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:37 2022] DELETE /identity/v3/users/29466df13aa540d085240203e26c74fa => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:37.259762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:37.260126 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:37.260468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:37.283445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:37.284759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:37.355612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1268/2508] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:37.365224 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:37.366740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:37.435234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3923d0-7ffe-45d5-9f1d-a4db185d70bf admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:37.439387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:37.440063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:37.440360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:37.440627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:37.453364 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:37.456132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:37.469776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1241/2509] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:37.828058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:37.830099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:37.856495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e4e84-3a32-40ae-9b38-5deb2bc7ec03 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:37.865813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:37.866893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:37.867244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:37.867608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] PATH_INFO: `/v3/projects/d45bc9a11cb14db9a3384262b5c9a37b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:37.869506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45bc9a11cb14db9a3384262b5c9a37b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:37.871774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:37.932490 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45bc9a11cb14db9a3384262b5c9a37b', 'request_id': 'req-bb3b7925-0230-4ce2-8f59-4d8eeec87636'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.943105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45bc9a11cb14db9a3384262b5c9a37b', 'request_id': 'req-bb3b7925-0230-4ce2-8f59-4d8eeec87636'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.953476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45bc9a11cb14db9a3384262b5c9a37b', 'request_id': 'req-bb3b7925-0230-4ce2-8f59-4d8eeec87636'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.967726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] The token cache is being invalidate because project d45bc9a11cb14db9a3384262b5c9a37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:37.968030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.968681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:37.972772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1269/2510] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:37 2022] DELETE /identity/v3/projects/d45bc9a11cb14db9a3384262b5c9a37b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:37.976690 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:37.987591 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf17ec-1b54-4e59-a4bd-cbe268a23276 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:38.063084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:38.063971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.064252 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.064526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] PATH_INFO: `/v3/users/6577735a658344f6a2228aab6d92ded7` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.065746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6577735a658344f6a2228aab6d92ded7)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:38.069681 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:38.137844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6577735a658344f6a2228aab6d92ded7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.145399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6577735a658344f6a2228aab6d92ded7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.146522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6577735a658344f6a2228aab6d92ded7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.147246 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6577735a658344f6a2228aab6d92ded7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.158379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3dfec1ca-e2d8-4bb3-999c-d86ca9537d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6577735a658344f6a2228aab6d92ded7'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.188225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1242/2511] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:37 2022] DELETE /identity/v3/users/6577735a658344f6a2228aab6d92ded7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:38.528228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:38.529791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:38.552559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b7925-0230-4ce2-8f59-4d8eeec87636 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:38.628013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:38.628883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.629170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.629446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] PATH_INFO: `/v3/projects/7979695395494dc9a7073952bd6699b4` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.630681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7979695395494dc9a7073952bd6699b4)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:38.633099 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:38.683365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7979695395494dc9a7073952bd6699b4', 'request_id': 'req-47fb3830-a6db-42d2-a793-a789bfb0c211'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.690235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7979695395494dc9a7073952bd6699b4', 'request_id': 'req-47fb3830-a6db-42d2-a793-a789bfb0c211'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.696943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7979695395494dc9a7073952bd6699b4', 'request_id': 'req-47fb3830-a6db-42d2-a793-a789bfb0c211'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.711213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] The token cache is being invalidate because project 7979695395494dc9a7073952bd6699b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:38.711653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.712459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-47fb3830-a6db-42d2-a793-a789bfb0c211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:38.717597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1270/2512] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:38 2022] DELETE /identity/v3/projects/7979695395494dc9a7073952bd6699b4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:38.719203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.719518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.719800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.741595 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:38.743716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:38.809240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1243/2513] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:38.817012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.817294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.817589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.838422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:38.839966 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:38.849811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1271/2514] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:38.853840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:38.863671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacb8dc-a4b2-4f6f-af3c-1f36f143475f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:38.870126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:38.870954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.871411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.871808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.872809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:38.873182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:38.874913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:38.888726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1244/2515] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 15:09:38.893317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:38.902984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396d1f8-3802-4574-b5ca-25ca0611ed8c None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:38.907820 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:38.908478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:38.908761 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:38.909035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:38.909912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:38.912003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:38.947000 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1272/2516] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:38 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:38.948380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:38.959539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba42c-41df-435e-86c4-8f8f4d134e13 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.019198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.020435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.020880 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.021346 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.022369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.024375 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.047048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1245/2517] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:38 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:39.051374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.063548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f647da-c9d4-4bc2-84d8-f9b074e6979b admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.068612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.070459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.070823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.071113 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.071829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.073615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.090133 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1273/2518] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:39.094180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.105117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137be1-5386-47f2-9919-75e68be3b70d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.110741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.111568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.111948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.112336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.113164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.114917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056acbcc-e585-4901-a136-9f9fc0a32a17 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.126667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1246/2519] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:39.130577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.143187 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b553-ecc3-4690-9512-fdc3710bd298 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.148419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.149334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.149626 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.149887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] PATH_INFO: `/v3/projects/458b33f9398c4e3b90a243d483440d70/users/512c7dc055bf4ef081abac8b3c2c403b/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.167042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=458b33f9398c4e3b90a243d483440d70, user_id=512c7dc055bf4ef081abac8b3c2c403b, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.169797 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.188488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1274/2520] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:39 2022] PUT /identity/v3/projects/458b33f9398c4e3b90a243d483440d70/users/512c7dc055bf4ef081abac8b3c2c403b/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:39.200286 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.200577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.200860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.230843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] MFA Rules not processed for user `512c7dc055bf4ef081abac8b3c2c403b`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:39.232219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:39.313121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1247/2521] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:39.317873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.394803 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187e65d-e168-4d4c-a7ca-cf0db497b5bc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.399945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.400685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.401047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.401403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.402171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.404184 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.431112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1275/2522] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:39 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:39.435398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.449936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cedba3-2900-4a36-a871-9d1111af8caa None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.535991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.536687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.536971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.537299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.538015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.541814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.564636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1248/2523] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:39 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:09:39.568504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.586854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608f1c5-0cad-4d9c-ba3f-f7162763e166 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.592390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.593098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.593385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.593659 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.594436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.596021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.611755 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1276/2524] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:39.615902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.627749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc158a-6eb7-4bb9-8599-7ca538dc3803 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.634299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.639859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.640844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.641340 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] PATH_INFO: `/v3/projects/ac36ce8d398d42a49608ea310af33884/users/0f5c7b2bf72c4ed883da5c1aaa03812a/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.662199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac36ce8d398d42a49608ea310af33884, user_id=0f5c7b2bf72c4ed883da5c1aaa03812a, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.665083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.681606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1249/2525] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:39 2022] PUT /identity/v3/projects/ac36ce8d398d42a49608ea310af33884/users/0f5c7b2bf72c4ed883da5c1aaa03812a/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:09:39.686675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.698941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7acb-dbdd-45ce-a04a-ece674e119c5 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.765672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.766598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.768041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.768862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.770175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.774304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437873-1770-4154-9144-f14fde5259f7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.786214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1277/2526] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:39.790974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.804933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b68a7-92c5-4e56-ae75-a4a5ffd6ca97 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:39.810415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:39.811299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.811572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.811845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] PATH_INFO: `/v3/projects/ac36ce8d398d42a49608ea310af33884/users/0f5c7b2bf72c4ed883da5c1aaa03812a/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.813786 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac36ce8d398d42a49608ea310af33884, user_id=0f5c7b2bf72c4ed883da5c1aaa03812a, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:39.818119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:39.831105 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1250/2527] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:39 2022] PUT /identity/v3/projects/ac36ce8d398d42a49608ea310af33884/users/0f5c7b2bf72c4ed883da5c1aaa03812a/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:39.841590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:39.841876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:39.842274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:39.875478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] MFA Rules not processed for user `0f5c7b2bf72c4ed883da5c1aaa03812a`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:39.876793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:39.957398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1278/2528] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:39.968776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:39.970685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:40.037312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459224a4-ee62-4fc9-8db1-7b01f1951dd3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:40.042471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:40.043160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:40.043444 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:40.043720 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:40.105532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:40.107208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:40.117313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1251/2529] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:44.252378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:44.253815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:44.265994 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabddff2-3c96-4e18-a9d2-3c758aa78654 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:44.269556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:44.270194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:44.270514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:44.270826 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:44.280703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:44.282426 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8907b-8c97-4a58-afd5-3005cc434fc3 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:44.291839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1279/2530] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:44.978477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:44.979776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:45.033514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1f879-af54-4190-a4a6-0731afc476a2 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:45.037199 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:45.037837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:45.038138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:45.038411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:45.040667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:45.050111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:45.051582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:45.060350 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1252/2531] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:46.812427 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:46.812923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:46.813031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:46.830910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:46.832089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:46.884274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1280/2532] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:46.896080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:46.897450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:46.970278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547206c7-a073-434c-9f5a-0a9af57b34ff service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:46.976116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:46.976765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:46.977042 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:46.977421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:46.986569 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:46.988765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637720f6-d5d3-45e1-a7f8-410f84b920b7 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:47.001243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1253/2533] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:09:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:09:49.182468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:49.252923 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13586261-94e0-4f49-9a4a-e1da5c29b8be None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:49.258096 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:49.259023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:49.259304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:49.259577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] PATH_INFO: `/v3/users/512c7dc055bf4ef081abac8b3c2c403b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:49.260824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512c7dc055bf4ef081abac8b3c2c403b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:49.262838 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:49.316686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512c7dc055bf4ef081abac8b3c2c403b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.317378 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512c7dc055bf4ef081abac8b3c2c403b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.325452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512c7dc055bf4ef081abac8b3c2c403b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.326207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512c7dc055bf4ef081abac8b3c2c403b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.335379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512c7dc055bf4ef081abac8b3c2c403b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.356876 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1281/2534] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:49 2022] DELETE /identity/v3/users/512c7dc055bf4ef081abac8b3c2c403b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:49.362273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:49.363022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:49.363022 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:49.380954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:49.382360 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:49.445593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1254/2535] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:49.455395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:49.456873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:49.523064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf178dfe-01ec-4077-a64e-476c9312a5d8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:49.529323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:49.530560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:49.530950 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:49.531326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:49.541619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:49.543216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:49.554147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1282/2536] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:49.843082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:49.844543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:49.860126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56b107-2908-40c3-97da-a36cffd76279 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:49.865301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:49.866165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:49.866721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:49.867161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] PATH_INFO: `/v3/projects/458b33f9398c4e3b90a243d483440d70` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:49.870084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=458b33f9398c4e3b90a243d483440d70)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:49.873944 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:49.937449 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458b33f9398c4e3b90a243d483440d70', 'request_id': 'req-303c784a-688b-4316-a079-f8a93bb87e0c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.943093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458b33f9398c4e3b90a243d483440d70', 'request_id': 'req-303c784a-688b-4316-a079-f8a93bb87e0c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.950195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458b33f9398c4e3b90a243d483440d70', 'request_id': 'req-303c784a-688b-4316-a079-f8a93bb87e0c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.966414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] The token cache is being invalidate because project 458b33f9398c4e3b90a243d483440d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:49.966733 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.967451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:49.972113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1255/2537] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:49 2022] DELETE /identity/v3/projects/458b33f9398c4e3b90a243d483440d70 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:49.973293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:49.984866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d8cc-925c-4825-a00c-3dcab5290341 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.046584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.047558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.047928 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.048270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] PATH_INFO: `/v3/users/0f5c7b2bf72c4ed883da5c1aaa03812a` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.049565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5c7b2bf72c4ed883da5c1aaa03812a)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.051635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.113890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5c7b2bf72c4ed883da5c1aaa03812a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.114641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5c7b2bf72c4ed883da5c1aaa03812a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.123185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5c7b2bf72c4ed883da5c1aaa03812a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.123869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5c7b2bf72c4ed883da5c1aaa03812a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.131951 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-84885842-24f1-4bc7-b182-deef263afec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5c7b2bf72c4ed883da5c1aaa03812a'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.151341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1283/2538] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:49 2022] DELETE /identity/v3/users/0f5c7b2bf72c4ed883da5c1aaa03812a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:50.448539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:50.450652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:50.482646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c784a-688b-4316-a079-f8a93bb87e0c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.542410 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.543298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.543582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.543863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] PATH_INFO: `/v3/projects/ac36ce8d398d42a49608ea310af33884` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.545076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac36ce8d398d42a49608ea310af33884)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.547023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.599077 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac36ce8d398d42a49608ea310af33884', 'request_id': 'req-96f24db7-7fcb-4a88-9876-3b7b4497bc94'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.605327 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac36ce8d398d42a49608ea310af33884', 'request_id': 'req-96f24db7-7fcb-4a88-9876-3b7b4497bc94'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.612972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac36ce8d398d42a49608ea310af33884', 'request_id': 'req-96f24db7-7fcb-4a88-9876-3b7b4497bc94'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.628491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] The token cache is being invalidate because project ac36ce8d398d42a49608ea310af33884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:50.628869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.629578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-96f24db7-7fcb-4a88-9876-3b7b4497bc94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:50.634303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1256/2539] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:50 2022] DELETE /identity/v3/projects/ac36ce8d398d42a49608ea310af33884 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:50.640058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.640527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.640657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.659410 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:50.660615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:50.720782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1284/2540] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:50.729967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.730311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.730827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.751785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:50.753093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:50.761525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1257/2541] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:50.766013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:50.776676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2f568-03a7-48f5-8e7c-503c1a70ff2d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.783275 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.783936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.784214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.784487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.785207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:50.785510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.787452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.800689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1285/2542] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:09:50.804893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:50.814149 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5021175-540e-4968-867c-d77cc8dd8b94 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.819451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.820408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.820672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.820963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.822138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.824244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.857096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1258/2543] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:50 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:50.861370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:50.873129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bdc1e-a52d-47a9-8f3b-27215aea69b1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.931274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.932010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.932301 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.932580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.933290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.935255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.956543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1286/2544] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:50 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:50.960630 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:50.973494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101909fc-b451-4d41-ad73-577f3d7e4f0a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:50.978935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:50.979628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:50.980011 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:50.980275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:50.981001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:50.982632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:50.994517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1259/2545] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:09:50.998698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:51.013810 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ea438-8211-46cb-a51a-25122ed09bc7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:51.018469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:51.019135 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.019421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.019698 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.020479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:51.022082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37232fec-54df-461a-b307-18196182f3e2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:51.033842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1287/2546] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:51.038017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:51.048050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce98b6-5396-4c96-a787-cf9941d1e63d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:51.052902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:51.053734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.054021 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.054298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] PATH_INFO: `/v3/projects/f99ec9221a3e493db85d4da129378ae3/users/2841511cfcd8447b87206726d670c6d9/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.070278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99ec9221a3e493db85d4da129378ae3, user_id=2841511cfcd8447b87206726d670c6d9, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:51.073501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:51.087766 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1260/2547] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:51 2022] PUT /identity/v3/projects/f99ec9221a3e493db85d4da129378ae3/users/2841511cfcd8447b87206726d670c6d9/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:51.099742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.100445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.101434 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.146849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] MFA Rules not processed for user `2841511cfcd8447b87206726d670c6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:51.148374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:51.236245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1288/2548] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:51.283450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:51.285511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:51.345988 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9811eb-7361-4c08-8d4d-2d7659fdc2a2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:51.349820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:51.350504 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.350783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.351051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.360090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:51.361529 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec1521-904d-413e-8838-c7813deb118d service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:51.371372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1261/2549] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:51.387339 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:51.453753 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a457a3-8d2b-486a-b71a-9aefa182516b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:51.458793 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:51.459747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.460068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.460354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] PATH_INFO: `/v3/users/2841511cfcd8447b87206726d670c6d9` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.461602 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2841511cfcd8447b87206726d670c6d9)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:51.463580 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:51.514452 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2841511cfcd8447b87206726d670c6d9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:51.515237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2841511cfcd8447b87206726d670c6d9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:51.522237 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2841511cfcd8447b87206726d670c6d9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:51.523009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2841511cfcd8447b87206726d670c6d9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:51.530815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2841511cfcd8447b87206726d670c6d9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:51.548854 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1289/2550] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:09:51 2022] DELETE /identity/v3/users/2841511cfcd8447b87206726d670c6d9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:51.553524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.554344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.554684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.573999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:51.575284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:51.641255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1262/2551] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:51.651562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:51.652980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:51.711439 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd23b-319f-4678-a44f-33e11669563a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:51.715305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:51.715964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:51.716253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:51.716536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:51.726539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:51.728526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90b3a7-b8f6-4309-8de5-c1163e92f8f2 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:51.738619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1290/2552] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:09:52.021287 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.022845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:52.035628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905fa29-27d4-4a16-a4fa-665bd15b8702 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.053041 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.053940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.054336 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.054656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] PATH_INFO: `/v3/projects/f99ec9221a3e493db85d4da129378ae3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.056477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99ec9221a3e493db85d4da129378ae3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.058510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.110967 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99ec9221a3e493db85d4da129378ae3', 'request_id': 'req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:52.116600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99ec9221a3e493db85d4da129378ae3', 'request_id': 'req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:52.123619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99ec9221a3e493db85d4da129378ae3', 'request_id': 'req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:52.137894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] The token cache is being invalidate because project f99ec9221a3e493db85d4da129378ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:09:52.138182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:52.138852 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-2abf76ff-27ca-4e7a-8c31-ca0de51feaca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:09:52.140417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1263/2553] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:09:52 2022] DELETE /identity/v3/projects/f99ec9221a3e493db85d4da129378ae3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:52.146979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.147215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.147480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.167436 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:52.168751 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:52.235747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1291/2554] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:52.243716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.244016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.244292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.268748 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:52.269964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:52.279484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1264/2555] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:09:52.284583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.295106 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7017-b0ca-42ce-b31f-280a8b86d619 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.308573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.309346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.309710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.310055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.310823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:09:52.311172 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.313032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.329748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1292/2556] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:09:52.331143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.343174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be37c9-7409-4a8b-b1b6-84c08271d712 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.350623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.351351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.351655 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.351990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.353464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.358378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.397383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1265/2557] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:52 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:52.402012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.413029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b06ac-db32-434d-813e-0cfbce867341 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.504482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.505412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.505885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.506291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.507162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.510459 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.534335 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1293/2558] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:52 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:52.548357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.558727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30540dab-9491-4b45-aaea-a926d314bece admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.567284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.568319 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.568770 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.569122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.570234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.572596 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.584414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1266/2559] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:52.589191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.599673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5406-48e9-4df4-8fe9-55a668a28dcc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.605508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.606195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.606508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.606787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.607519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.609117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b3f13-79be-40e9-9681-4ea2d2951f99 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.623149 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1294/2560] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:52.624626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.636230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806f015-8744-4394-9ead-ba7fb695a33f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.641266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.642176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.642477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.642756 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] PATH_INFO: `/v3/projects/69cc7901bd8e4ff68b7e943121e69580/users/4db1da4361c444da87400db06bbae3e4/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.660088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69cc7901bd8e4ff68b7e943121e69580, user_id=4db1da4361c444da87400db06bbae3e4, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.663470 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.678903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1267/2561] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:52 2022] PUT /identity/v3/projects/69cc7901bd8e4ff68b7e943121e69580/users/4db1da4361c444da87400db06bbae3e4/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:52.683954 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add99c1c-75ba-4014-8b56-d9f76beafbe1 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.684514 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add99c1c-75ba-4014-8b56-d9f76beafbe1 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.684623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add99c1c-75ba-4014-8b56-d9f76beafbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.707589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-add99c1c-75ba-4014-8b56-d9f76beafbe1 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:52.708829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-add99c1c-75ba-4014-8b56-d9f76beafbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:52.770509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1295/2562] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:52.781503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:52.783230 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:52.848436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632a95-65e4-49ad-a1c7-4436a3a1b642 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:52.852126 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:52.852822 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:52.853080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:52.853351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:52.863155 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:52.864608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:52.875268 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1268/2563] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:09:57.725242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:57.725825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:57.725825 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:57.757318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] MFA Rules not processed for user `4db1da4361c444da87400db06bbae3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:09:57.758845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:09:57.826268 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1296/2564] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:09:57.830862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:57.840791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499b98d-e9bb-41aa-84e5-ab30ecf04ed7 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:57.845443 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:57.846104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:57.846409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:57.846701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:57.847441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:57.849362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:57.877406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1269/2565] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:09:57 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:57.881482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:57.892544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf005-ad63-436c-a36a-d18a95fa3d2d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:57.951811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:57.952488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:57.952768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:57.953040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:57.953743 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:57.955690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:57.980984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1297/2566] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:09:57 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:09:57.982599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:57.995016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da024e-ad1f-4a70-ab96-a5427b4c9b8f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:58.000394 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:58.001071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:58.001353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:58.001629 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:58.002357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:58.004000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:58.015366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1270/2567] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:09:58.019485 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:58.029182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33e63-9aab-4edd-a69e-9133caefbe04 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:58.034055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:58.034965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:58.035242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:58.035509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] PATH_INFO: `/v3/projects/657130c9752e4748817a4bafcd53f33d/users/36904454aa124cf1b3eaf9b15d65c8ab/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:58.050794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=657130c9752e4748817a4bafcd53f33d, user_id=36904454aa124cf1b3eaf9b15d65c8ab, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:58.053438 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:58.066821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1298/2568] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:58 2022] PUT /identity/v3/projects/657130c9752e4748817a4bafcd53f33d/users/36904454aa124cf1b3eaf9b15d65c8ab/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:09:58.070729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:58.080918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc85b-ca5a-42d9-9744-7f73fe464a0a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:58.142487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:58.143182 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:58.143548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:58.143727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:58.144487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:58.146069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fcea4-e995-4097-b2d1-fd51a4af4657 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:58.156351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1271/2569] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:09:58.160265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:09:58.170991 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f44215-86a6-4020-99fd-fda177bda8fa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:09:58.175692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:09:58.176562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:09:58.176892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:09:58.177163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] PATH_INFO: `/v3/projects/657130c9752e4748817a4bafcd53f33d/users/36904454aa124cf1b3eaf9b15d65c8ab/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:09:58.179129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=657130c9752e4748817a4bafcd53f33d, user_id=36904454aa124cf1b3eaf9b15d65c8ab, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:09:58.181864 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:09:58.191723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1299/2570] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:09:58 2022] PUT /identity/v3/projects/657130c9752e4748817a4bafcd53f33d/users/36904454aa124cf1b3eaf9b15d65c8ab/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:10:03.134534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102128f-ebdd-4801-8a79-631cc9975508 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:03.135136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102128f-ebdd-4801-8a79-631cc9975508 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:03.135136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102128f-ebdd-4801-8a79-631cc9975508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:03.168100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e102128f-ebdd-4801-8a79-631cc9975508 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:03.178539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e102128f-ebdd-4801-8a79-631cc9975508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:03.270934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1272/2571] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:03.279215 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:03.281279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:03.408089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004f8a-7230-41d5-be66-9bc1d3d2a89c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:03.417784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:03.418882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:03.419425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:03.419868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:03.430892 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:03.432781 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:03.481628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1300/2572] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:10:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:10:03.691835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68257-5fbc-4d61-beca-602ac33e2f2e None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:03.692127 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68257-5fbc-4d61-beca-602ac33e2f2e None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:03.692406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68257-5fbc-4d61-beca-602ac33e2f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:03.727250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-94e68257-5fbc-4d61-beca-602ac33e2f2e None None] MFA Rules not processed for user `36904454aa124cf1b3eaf9b15d65c8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:03.728594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-94e68257-5fbc-4d61-beca-602ac33e2f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:03.842406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1273/2573] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:03.851800 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:03.853509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:03.995863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df091f0-a97f-4736-ad5e-583b65e9078d service swift] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:04.003159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1bcef95380344929f010f445efb1373', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:04.003847 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:04.006274 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:04.006545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:04.110926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:04.111835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:04.123046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1301/2574] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:04.274380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170301bf-9457-4da4-88cc-b32a8bd63cc8 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:04.274998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170301bf-9457-4da4-88cc-b32a8bd63cc8 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:04.274998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170301bf-9457-4da4-88cc-b32a8bd63cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:04.293667 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-170301bf-9457-4da4-88cc-b32a8bd63cc8 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:04.295015 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-170301bf-9457-4da4-88cc-b32a8bd63cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:04.303437 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1274/2575] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:04.317927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:04.331651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d2d0b-ac34-40c5-a658-68217e67e107 service glance] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:04.339770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:04.340431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:04.340707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:04.341127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:04.350005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:04.352374 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:04.363075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1302/2576] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:10:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:10:13.614161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536813c-55fb-4bd1-8a39-75ad9ed7e228 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:13.614671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536813c-55fb-4bd1-8a39-75ad9ed7e228 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:13.614767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536813c-55fb-4bd1-8a39-75ad9ed7e228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:13.639306 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-5536813c-55fb-4bd1-8a39-75ad9ed7e228 None None] MFA Rules not processed for user `4db1da4361c444da87400db06bbae3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:13.640688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-5536813c-55fb-4bd1-8a39-75ad9ed7e228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:13.650790 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1275/2577] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:14.684444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:14.741252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca735de-3786-43f9-a64b-8aea8d25540b service swift] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:14.744819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:14.745455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:14.745736 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:14.746009 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:14.748071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:14.807500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:14.808916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:14.818332 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1303/2578] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:15.321175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:15.321652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:15.321749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:15.339409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:15.340604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:15.347696 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1276/2579] 10.222.0.83 () {58 vars in 1009 bytes} [Tue May 10 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:15.359979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:15.371676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f30502-050c-4bcd-afdd-9b6e8f1d5d40 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:15.376735 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:15.377375 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:15.377655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:15.377933 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:15.387064 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:15.388690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:15.397055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1304/2580] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:10:19.375213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:19.376773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:19.393523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604bdd1-0628-4669-9d5d-0bd6b3666ca6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:19.402511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:19.402511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:19.402511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:19.402511 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] PATH_INFO: `/v3/users/06701ca6a5e0427c9d5b540133c522a0` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:19.403430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06701ca6a5e0427c9d5b540133c522a0)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:19.405669 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:19.456031 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06701ca6a5e0427c9d5b540133c522a0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.465377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06701ca6a5e0427c9d5b540133c522a0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.466377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06701ca6a5e0427c9d5b540133c522a0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.467314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06701ca6a5e0427c9d5b540133c522a0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.482027 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ea11ff32-2c08-45de-b209-41fd570eeb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06701ca6a5e0427c9d5b540133c522a0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.499815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1277/2581] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:19 2022] DELETE /identity/v3/users/06701ca6a5e0427c9d5b540133c522a0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:19.687051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:19.699578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3009c-1f48-4f54-998a-cbf767b65a55 service swift] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:19.761484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:19.762346 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:19.762664 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:19.762943 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] PATH_INFO: `/v3/projects/8961e0a1d6d443a0847533b83aa1ea8b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:19.764128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8961e0a1d6d443a0847533b83aa1ea8b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:19.766066 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:19.831246 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8961e0a1d6d443a0847533b83aa1ea8b', 'request_id': 'req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.837605 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8961e0a1d6d443a0847533b83aa1ea8b', 'request_id': 'req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.843176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8961e0a1d6d443a0847533b83aa1ea8b', 'request_id': 'req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.859507 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] The token cache is being invalidate because project 8961e0a1d6d443a0847533b83aa1ea8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:19.859977 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.860675 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-3f21b2ce-1b90-41e9-a35e-a76ddd7c1c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:19.865789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1305/2582] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:19 2022] DELETE /identity/v3/projects/8961e0a1d6d443a0847533b83aa1ea8b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:19.868356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:19.868723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:19.869074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:19.887856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:19.889157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:19.955099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1278/2583] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:19.956666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:19.956981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:19.957251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:19.977895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:19.979273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:19.987832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1306/2584] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:19.992163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.002472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1a977-3e82-47d3-9e7d-919f04a282b3 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.007663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.008441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.008820 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.009305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.010136 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:10:20.010689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.012398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.025694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1279/2585] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:10:20.030010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.040027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a3d0-38c2-436e-9dcd-40c8f72aca9a None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.045318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.046076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.046356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.046759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.047458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.049728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.081140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1307/2586] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:20.085906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.096066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-a5b7-428c-96d9-d49be8b9d4ad admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.154864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.155555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.155862 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.156137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.156821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.158800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.181506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1280/2587] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:20.182940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.193515 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84de9d-7674-4cc6-8a4e-4fe84464a070 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.198779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.199496 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.199874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.200344 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.201142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.202896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.215456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1308/2588] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:20.219409 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.230163 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baf31a-71ba-4172-8e21-b0b730a9b21d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.234872 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.235547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.235815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.236090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.236810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.238524 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd1902-fc4c-41d7-b2cc-4c35b3586da6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.252723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1281/2589] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:10:20.254419 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.265470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb5ef8-f963-4929-90f2-bc2428b4303a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.271544 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.273397 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.273866 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.274313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] PATH_INFO: `/v3/projects/907d3b9fe9cf420999c3ae3070766e2c/users/224004b773134a14ad576fd9b0ee4a65/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.289922 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=907d3b9fe9cf420999c3ae3070766e2c, user_id=224004b773134a14ad576fd9b0ee4a65, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.292594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.305305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1309/2590] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:20 2022] PUT /identity/v3/projects/907d3b9fe9cf420999c3ae3070766e2c/users/224004b773134a14ad576fd9b0ee4a65/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:20.315393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.315688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.315968 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.345120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] MFA Rules not processed for user `224004b773134a14ad576fd9b0ee4a65`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:20.346331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:20.433401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1282/2591] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:20.437539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.517294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707091b5-bdd3-4757-aa28-0c7a901afe99 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.522142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.523415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.523714 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.523979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.524689 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.526620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.555476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1310/2592] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:20.560528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.574698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d551e6-b716-4e7a-8391-9b2368eacef8 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.636298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.637590 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.638078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.638643 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.639545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.641704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.665447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1283/2593] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/users => generated 308 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:20.667425 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.677456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8268-13c7-43f6-8d50-8a4ae9d6d3a3 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.682539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.683186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.683467 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.683741 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.684458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.686036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.696706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1311/2594] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:20.700800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.712272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2a54-7eeb-4fd6-b894-04e9686b2f3d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.717007 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.717902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.718213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.718503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] PATH_INFO: `/v3/projects/277bf9fa64ab422f955de7af66c1004d/users/79555e5af10149559874844ce9790b66/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.734641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] RBAC: Authorizing `identity:create_grant(project_id=277bf9fa64ab422f955de7af66c1004d, user_id=79555e5af10149559874844ce9790b66, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.737440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.752668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1284/2595] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:20 2022] PUT /identity/v3/projects/277bf9fa64ab422f955de7af66c1004d/users/79555e5af10149559874844ce9790b66/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:10:20.756606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.768128 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdb2-0033-4da5-8840-c224722e9aef admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.831752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.832422 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.832705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.832996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.833691 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.835380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c010dd-488d-4caa-8b9f-8c463a1fc029 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.848462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1312/2596] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:20.849441 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:20.860003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d54142-45e4-4245-99ff-54978d40169a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:20.864856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:20.865740 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.866016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.866282 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] PATH_INFO: `/v3/projects/277bf9fa64ab422f955de7af66c1004d/users/79555e5af10149559874844ce9790b66/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.868859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=277bf9fa64ab422f955de7af66c1004d, user_id=79555e5af10149559874844ce9790b66, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:20.871543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:20.882838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1285/2597] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:20 2022] PUT /identity/v3/projects/277bf9fa64ab422f955de7af66c1004d/users/79555e5af10149559874844ce9790b66/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:10:20.894130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:20.894445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:20.894725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:20.921779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] MFA Rules not processed for user `79555e5af10149559874844ce9790b66`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:20.923248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:21.003259 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1313/2598] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:21.011919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:21.014226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:21.072902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e1c-9491-47d6-9e49-f95e57054abb admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:21.076821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:21.077545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:21.077900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:21.078244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:21.088408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:21.089984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340e3a3-50fa-45a2-bef6-3ddfbadea6b7 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:21.099908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1286/2599] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:21.254255 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:21.318021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b430-8564-4af2-816f-4c7a7d33f52b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:21.322817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:21.323795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:21.324156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:21.324612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] PATH_INFO: `/v3/users/224004b773134a14ad576fd9b0ee4a65` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:21.325904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] RBAC: Authorizing `identity:delete_user(user_id=224004b773134a14ad576fd9b0ee4a65)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:21.327955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:21.377953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224004b773134a14ad576fd9b0ee4a65'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:21.378697 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224004b773134a14ad576fd9b0ee4a65'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:21.387175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224004b773134a14ad576fd9b0ee4a65'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:21.388026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224004b773134a14ad576fd9b0ee4a65'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:21.396360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224004b773134a14ad576fd9b0ee4a65'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:21.418475 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1314/2600] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:21 2022] DELETE /identity/v3/users/224004b773134a14ad576fd9b0ee4a65 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:21.423039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:21.423362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:21.423694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:21.445099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:21.446313 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:21.516929 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1287/2601] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:21.529540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:21.531116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:21.593478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511513b4-2759-4079-b4bd-abc3aff1f09d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:21.597370 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:21.598768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:21.598768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:21.598768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:21.609240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:21.610713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:21.621533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1315/2602] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:21.943173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:21.945197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:21.964083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25d4e-7c8e-4f0b-9c76-fec6070d249c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:21.968799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:21.969749 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:21.970157 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:21.970565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] PATH_INFO: `/v3/projects/907d3b9fe9cf420999c3ae3070766e2c` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:21.972044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=907d3b9fe9cf420999c3ae3070766e2c)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:21.974054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:22.024243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '907d3b9fe9cf420999c3ae3070766e2c', 'request_id': 'req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.029446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '907d3b9fe9cf420999c3ae3070766e2c', 'request_id': 'req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.035500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '907d3b9fe9cf420999c3ae3070766e2c', 'request_id': 'req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.047332 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] The token cache is being invalidate because project 907d3b9fe9cf420999c3ae3070766e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:22.047613 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.048238 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.049899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1288/2603] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:21 2022] DELETE /identity/v3/projects/907d3b9fe9cf420999c3ae3070766e2c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:22.053869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:22.063887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b788b0-a99b-4256-b1c5-67b0a39a94bf service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:22.121802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:22.122984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.123270 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.123543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] PATH_INFO: `/v3/users/79555e5af10149559874844ce9790b66` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.124740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79555e5af10149559874844ce9790b66)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:22.126702 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:22.175568 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79555e5af10149559874844ce9790b66'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.176243 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79555e5af10149559874844ce9790b66'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.183655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79555e5af10149559874844ce9790b66'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.184323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79555e5af10149559874844ce9790b66'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.191877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-280046d8-30cb-42b4-b836-4992155d9392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79555e5af10149559874844ce9790b66'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.210044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1316/2604] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:22 2022] DELETE /identity/v3/users/79555e5af10149559874844ce9790b66 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:22.489236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:22.490830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:22.506727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5ae75-7ffc-4aa1-b51d-8068ded9b240 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:22.614096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:22.617932 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.618884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.619893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] PATH_INFO: `/v3/projects/277bf9fa64ab422f955de7af66c1004d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.622918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277bf9fa64ab422f955de7af66c1004d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:22.626877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:22.717984 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277bf9fa64ab422f955de7af66c1004d', 'request_id': 'req-63aa4b8f-3b08-48f5-809d-294772b95fd3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.742175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277bf9fa64ab422f955de7af66c1004d', 'request_id': 'req-63aa4b8f-3b08-48f5-809d-294772b95fd3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.750169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277bf9fa64ab422f955de7af66c1004d', 'request_id': 'req-63aa4b8f-3b08-48f5-809d-294772b95fd3'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.767134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] The token cache is being invalidate because project 277bf9fa64ab422f955de7af66c1004d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:22.768035 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.768183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-63aa4b8f-3b08-48f5-809d-294772b95fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:22.770147 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1289/2605] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:22 2022] DELETE /identity/v3/projects/277bf9fa64ab422f955de7af66c1004d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:22.775350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.775766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.776137 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.796669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:22.798131 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:22.873872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1317/2606] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:22.875989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.876359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.876641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.895073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:22.896279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:22.904802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1290/2607] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:22.909726 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:22.926041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f1b52-8771-49b6-82ee-38fe9c530a6f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:22.930955 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:22.931754 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.932045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.932321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.933044 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:10:22.933352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:22.935025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:22.947284 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1318/2608] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:10:22.951670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:22.961352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab7932-7e6e-4846-98f7-49fc1dc5f6da None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:22.965956 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:22.966805 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:22.967094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:22.967426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:22.968782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:22.970377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.006034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1291/2609] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:22 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:23.006595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.017002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730beef-90df-4756-8257-2904faa151c8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.074600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.075319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.075545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.075970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.076713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.078663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.098075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1319/2610] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:23 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:10:23.102608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.113473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0884e1-69c5-410b-9a89-c52c82981bdf admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.119071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.119746 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.120028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.120303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.121030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.122948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.133830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1292/2611] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:23.137990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.148350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c192c4-4eb8-420c-904f-6e1b17420505 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.152879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.153526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.153811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.154088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.154837 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.156413 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc91da-bd3d-41c2-a640-09ae8a7c2d17 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.167466 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1320/2612] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:23.171603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.182100 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bca23e-7223-4fbb-b182-7ca42f5cc0d5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.186991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.187892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.188193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.188457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] PATH_INFO: `/v3/projects/4367e7fb5570459ab658caeed328bb6f/users/279b1497f75447c88ce961465b724ad5/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.204921 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4367e7fb5570459ab658caeed328bb6f, user_id=279b1497f75447c88ce961465b724ad5, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.207686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.221137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1293/2613] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:23 2022] PUT /identity/v3/projects/4367e7fb5570459ab658caeed328bb6f/users/279b1497f75447c88ce961465b724ad5/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:23.232913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.233201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.233480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.262710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] MFA Rules not processed for user `279b1497f75447c88ce961465b724ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:23.263924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:23.329823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1321/2614] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:10:23.334614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.403483 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cba0f-bd37-4f63-acd1-1bb313337947 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.408750 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.409427 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.409710 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.409985 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.410795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.412708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.442498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1294/2615] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:23 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:23.446932 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.457402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba263e-cc79-45f7-b1be-73511b4af5ce None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.514718 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.515393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.515676 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.515953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.516673 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.518670 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.538723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1322/2616] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:23 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:10:23.542877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.553723 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095cb3-d10c-4c57-9db8-c7bbbd936b36 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.558757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.559732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.560104 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.560517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.561348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.563095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.574776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1295/2617] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:23.579025 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.590584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4850297-2bcb-485c-8535-5e5ba871c7ff admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.596019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.596924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.597208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.597478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] PATH_INFO: `/v3/projects/a493467e15f74de8ac35e0577e861d79/users/0fb4865a02ed428e81b6f484db532816/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.612808 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a493467e15f74de8ac35e0577e861d79, user_id=0fb4865a02ed428e81b6f484db532816, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.615483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.630672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1323/2618] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:23 2022] PUT /identity/v3/projects/a493467e15f74de8ac35e0577e861d79/users/0fb4865a02ed428e81b6f484db532816/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:23.635278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.645258 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a37f9-fad4-4495-94df-6259de9a2db4 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.703468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.704152 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.704431 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.704703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.705416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.707033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba439708-b8cb-4b04-94eb-379a672ca1c9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.717179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1296/2619] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:23.721444 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.732169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24530-3dfd-45aa-8108-cbfbf9b5e940 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.737886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.738830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.739118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.739453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] PATH_INFO: `/v3/projects/a493467e15f74de8ac35e0577e861d79/users/0fb4865a02ed428e81b6f484db532816/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.742034 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a493467e15f74de8ac35e0577e861d79, user_id=0fb4865a02ed428e81b6f484db532816, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.744750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.756071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1324/2620] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:23 2022] PUT /identity/v3/projects/a493467e15f74de8ac35e0577e861d79/users/0fb4865a02ed428e81b6f484db532816/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:10:23.768112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.768402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.768683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.795877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] MFA Rules not processed for user `0fb4865a02ed428e81b6f484db532816`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:23.797090 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:23.878648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1297/2621] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:10:23.894500 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:23.896858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:23.957727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68201ac-cdad-419d-ab51-62acbadbbad2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:23.962431 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:23.963770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:23.963770 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:23.964010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:23.973910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:23.975446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:23.986665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1325/2622] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:24.127357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:24.144372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982b459-bebf-4826-9d54-95e1a512f6a7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:24.147919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:24.148586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:24.148938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:24.149284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:24.215886 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:24.217461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828bbd6-2c8e-4fde-b0ab-39e0a49cacbc service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:24.228112 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1298/2623] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:10:27.004090 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:27.062748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645cc6a-d15a-4ebe-8426-bcb254284a48 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:27.067264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:27.068118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:27.068402 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:27.068680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] PATH_INFO: `/v3/users/279b1497f75447c88ce961465b724ad5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:27.070399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=279b1497f75447c88ce961465b724ad5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:27.072309 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:27.116921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279b1497f75447c88ce961465b724ad5'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.117560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279b1497f75447c88ce961465b724ad5'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.124262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279b1497f75447c88ce961465b724ad5'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.124879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279b1497f75447c88ce961465b724ad5'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.131537 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279b1497f75447c88ce961465b724ad5'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.148557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1326/2624] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:27 2022] DELETE /identity/v3/users/279b1497f75447c88ce961465b724ad5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:27.152938 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:27.153474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:27.153474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:27.170301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:27.171501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:27.228326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1299/2625] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:27.237759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:27.239158 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:27.295264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc299d-b944-43d8-8819-c8f7827c9e4e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:27.298895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:27.299536 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:27.299814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:27.300081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:27.308925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:27.310330 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:27.320076 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1327/2626] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:27.707691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:27.709177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:27.737213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea499e-10f4-4c57-9c55-74d41300b4da None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:27.744879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:27.745776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:27.746083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:27.746361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] PATH_INFO: `/v3/projects/4367e7fb5570459ab658caeed328bb6f` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:27.747943 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4367e7fb5570459ab658caeed328bb6f)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:27.749851 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:27.802658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4367e7fb5570459ab658caeed328bb6f', 'request_id': 'req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.809020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4367e7fb5570459ab658caeed328bb6f', 'request_id': 'req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.815080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4367e7fb5570459ab658caeed328bb6f', 'request_id': 'req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.827276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] The token cache is being invalidate because project 4367e7fb5570459ab658caeed328bb6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:27.827593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.828210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.829782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1300/2627] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:27 2022] DELETE /identity/v3/projects/4367e7fb5570459ab658caeed328bb6f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:27.833686 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:27.843157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1bff-bde0-45cd-8a9b-052ea4113064 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:27.897880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:27.898778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:27.899062 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:27.899338 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] PATH_INFO: `/v3/users/0fb4865a02ed428e81b6f484db532816` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:27.900541 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb4865a02ed428e81b6f484db532816)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:27.902470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:27.946727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb4865a02ed428e81b6f484db532816'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.947371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb4865a02ed428e81b6f484db532816'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.953827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb4865a02ed428e81b6f484db532816'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.954551 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb4865a02ed428e81b6f484db532816'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.961575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e2be53bc-2727-4658-8c26-ff675d02e170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb4865a02ed428e81b6f484db532816'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:27.978359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1328/2628] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:27 2022] DELETE /identity/v3/users/0fb4865a02ed428e81b6f484db532816 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:28.237833 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.239301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:28.253640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c19c7-fcd3-4462-bd12-da23dd0fd0e6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.308489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.309345 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.309628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.309903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] PATH_INFO: `/v3/projects/a493467e15f74de8ac35e0577e861d79` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.311091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a493467e15f74de8ac35e0577e861d79)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.313302 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.356386 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a493467e15f74de8ac35e0577e861d79', 'request_id': 'req-036d0b43-835b-4b2d-9246-e56bfddc7b4f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:28.362604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a493467e15f74de8ac35e0577e861d79', 'request_id': 'req-036d0b43-835b-4b2d-9246-e56bfddc7b4f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:28.368606 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a493467e15f74de8ac35e0577e861d79', 'request_id': 'req-036d0b43-835b-4b2d-9246-e56bfddc7b4f'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:28.381254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] The token cache is being invalidate because project a493467e15f74de8ac35e0577e861d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:28.381542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:28.382254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-036d0b43-835b-4b2d-9246-e56bfddc7b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:28.383821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1301/2629] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:28 2022] DELETE /identity/v3/projects/a493467e15f74de8ac35e0577e861d79 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:28.388696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.389214 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.389321 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.406782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:28.407979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:28.464354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1329/2630] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:28.472096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.472401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.472675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.490878 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:28.491997 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:28.499986 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1302/2631] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:28.503930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.513553 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadac54-6f6b-4fa6-aae1-85974db60f8d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.517949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.518623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.518902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.519169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.519873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:10:28.520169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.521723 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.533115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1330/2632] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:10:28.537131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.546217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0722d-8261-4c6f-a0eb-4932dee035d7 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.550893 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.551532 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.551818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.552094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.552807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.554742 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.579072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1303/2633] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:28 2022] POST /identity/v3/projects => generated 383 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:28.583019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.594981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1423b-0576-4d3d-860a-30a570b4b848 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.655802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.656944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.657311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.657608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.658611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.660527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.680177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1331/2634] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:28 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:10:28.684500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.694903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8766-e7af-4a01-b2ee-305d29a6bbd5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.699381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.700034 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.700311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.700580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.701283 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.702890 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.714413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1304/2635] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:28.718352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.727924 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b29b90-da18-4a91-a2c5-90802c965736 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.732756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.733421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.733707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.733983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.734738 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.736315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e0ab-ca12-44d5-bad6-c790aee973d8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.746138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1332/2636] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:10:28.750381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.760450 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be848b0-94f5-487b-aca3-6a6d14a04151 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.764962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.765816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.766094 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.766358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] PATH_INFO: `/v3/projects/12245f8834754fdbb646a0ebfe737444/users/3196703e28624f79b8639374727f8dab/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.780640 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12245f8834754fdbb646a0ebfe737444, user_id=3196703e28624f79b8639374727f8dab, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.783277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.795915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1305/2637] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:28 2022] PUT /identity/v3/projects/12245f8834754fdbb646a0ebfe737444/users/3196703e28624f79b8639374727f8dab/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:28.805628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.805921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.806312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.832462 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] MFA Rules not processed for user `3196703e28624f79b8639374727f8dab`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:28.833656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:28.896161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1333/2638] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:28.907253 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:28.909218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:28.966948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ac696-be6b-4fac-a026-9863524802a5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:28.970526 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:28.971166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:28.971448 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:28.971721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:28.979729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:28.981138 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6d6f9-5143-42ae-ac7e-1878a498acc5 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:28.989783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1306/2639] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:31.562559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:31.623297 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ef4f-ad6a-4319-a873-1a19a6182f76 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:31.627786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:31.628630 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:31.628914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:31.629191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] PATH_INFO: `/v3/users/3196703e28624f79b8639374727f8dab` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:31.630366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3196703e28624f79b8639374727f8dab)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:31.632807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:31.676904 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3196703e28624f79b8639374727f8dab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:31.677555 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3196703e28624f79b8639374727f8dab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:31.684082 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3196703e28624f79b8639374727f8dab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:31.684703 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3196703e28624f79b8639374727f8dab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:31.691435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3196703e28624f79b8639374727f8dab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:31.708317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1334/2640] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:31 2022] DELETE /identity/v3/users/3196703e28624f79b8639374727f8dab => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:31.713338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:31.713801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:31.713961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:31.730715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:31.731891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:31.788003 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1307/2641] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:10:31.797504 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:31.798862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:31.853010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6ba2-91fc-4cf2-8d27-c69f6d3fcd65 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:31.856590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:31.857222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:31.857506 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:31.857782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:31.867169 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:31.868583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00f819-d31f-4433-9ec2-018fa4411644 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:31.878060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1335/2642] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:32.134555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.136053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:32.147969 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1023-6880-4f7a-8b87-636ee80ba7d6 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.152517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.153461 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.154173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.154173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] PATH_INFO: `/v3/projects/12245f8834754fdbb646a0ebfe737444` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.155322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12245f8834754fdbb646a0ebfe737444)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.157279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.201482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12245f8834754fdbb646a0ebfe737444', 'request_id': 'req-eea0fcd0-5322-4cae-b893-2406fac4bec5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:32.206686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12245f8834754fdbb646a0ebfe737444', 'request_id': 'req-eea0fcd0-5322-4cae-b893-2406fac4bec5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:32.212666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12245f8834754fdbb646a0ebfe737444', 'request_id': 'req-eea0fcd0-5322-4cae-b893-2406fac4bec5'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:32.224434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] The token cache is being invalidate because project 12245f8834754fdbb646a0ebfe737444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:32.224716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:32.225325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-eea0fcd0-5322-4cae-b893-2406fac4bec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:32.227638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1308/2643] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:32 2022] DELETE /identity/v3/projects/12245f8834754fdbb646a0ebfe737444 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:32.232310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.232601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.232880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.250508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:32.251684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:32.307756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1336/2644] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:32.316684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.316990 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.317265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.334913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:32.336092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:32.344430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1309/2645] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:32.348695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.358680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a03cf4-37de-4d9e-8419-9c5545198b76 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.363247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.363897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.364179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.364457 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.365180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:10:32.365486 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.367769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.379760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1337/2646] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:10:32.384438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.393493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc92996-b6a5-47ba-ba93-4d8dc8399769 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.398751 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.399405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.399689 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.400080 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.400784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.402709 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.429802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1310/2647] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:32 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:32.434020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.443907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfea4f-6729-47c2-9896-da6acf95d8d6 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.498828 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.499487 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.499768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.500091 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.500802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.502694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.521916 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1338/2648] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:32 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:10:32.526331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.536098 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab12ee-0e8f-4356-82bd-185b4da564f6 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.540608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.541247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.541548 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.541818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.542552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.544120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.553927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1311/2649] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:32.557581 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.567565 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bceb8d-b78f-47da-9365-f9afecf50970 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.572638 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.573286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.573567 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.573842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.574620 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.576290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ccbab-5a40-4ce2-a041-ca0284dd14b2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.586508 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1339/2650] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:32.591106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.600759 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dcc48-1134-4b71-ae92-2fc28e527890 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.605228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.606083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.606362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.606660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] PATH_INFO: `/v3/projects/870b2a5de87144f58bdd6aebe78a3e47/users/d4fb5382ce0d4e3694f4c434afc39d94/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.620827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=870b2a5de87144f58bdd6aebe78a3e47, user_id=d4fb5382ce0d4e3694f4c434afc39d94, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.623799 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.637091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1312/2651] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:32 2022] PUT /identity/v3/projects/870b2a5de87144f58bdd6aebe78a3e47/users/d4fb5382ce0d4e3694f4c434afc39d94/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:10:32.642057 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38b87-f434-4ebc-8e0e-9504b09ffca5 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.642371 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38b87-f434-4ebc-8e0e-9504b09ffca5 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.642674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38b87-f434-4ebc-8e0e-9504b09ffca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.659710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6cb38b87-f434-4ebc-8e0e-9504b09ffca5 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:32.660894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6cb38b87-f434-4ebc-8e0e-9504b09ffca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:32.717695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1340/2652] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:32.727940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:32.729688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:32.783885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f1e92-0dc4-4615-917c-b5385f8b8f31 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:32.787567 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:32.788203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:32.788487 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:32.788831 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:32.798212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:32.799688 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:32.809641 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1313/2653] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:33.483435 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a239bc-d021-457d-ad54-da1d9485b306 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:33.483968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a239bc-d021-457d-ad54-da1d9485b306 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:33.484256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a239bc-d021-457d-ad54-da1d9485b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:33.514447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-70a239bc-d021-457d-ad54-da1d9485b306 None None] MFA Rules not processed for user `d4fb5382ce0d4e3694f4c434afc39d94`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:33.515762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-70a239bc-d021-457d-ad54-da1d9485b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:33.588513 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1341/2654] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:33.601399 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:33.602844 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:33.660536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b137a-4dce-4fd6-a33e-3ef5c653bf07 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:33.665692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:33.666565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:33.666917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:33.667197 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:33.675701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:33.677130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:33.685920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1314/2655] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:35.017153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:35.017652 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:35.017749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:35.034647 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] MFA Rules not processed for user `d4fb5382ce0d4e3694f4c434afc39d94`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:35.035832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:35.042964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1342/2656] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:45.824764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:45.826454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:45.839123 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a40a2-7f29-4cf7-8df9-e9c2f8d97ab3 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:45.842732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:45.843366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:45.843645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:45.843917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:45.845654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:45.857546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:45.859016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:45.869184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1315/2657] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:47.060308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:47.071016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907bf17-c1e7-4464-b224-d9377a2aaf49 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:47.077455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:47.078132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:47.078447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:47.078728 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:47.080552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:47.144013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:47.145461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:47.159463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1343/2658] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:49.984794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:49.987309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:50.002177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2708-ad9b-4a86-b850-a1d1261c6159 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.012972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.019251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.019451 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.019668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] PATH_INFO: `/v3/users/aa56c83ab8494901b85d00d4f592e5ce` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.021309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa56c83ab8494901b85d00d4f592e5ce)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.024033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.097454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa56c83ab8494901b85d00d4f592e5ce'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.105594 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa56c83ab8494901b85d00d4f592e5ce'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.106423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa56c83ab8494901b85d00d4f592e5ce'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.107277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa56c83ab8494901b85d00d4f592e5ce'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.125254 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-42be8a4d-6033-4333-b47f-5f808e0e156f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa56c83ab8494901b85d00d4f592e5ce'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.148773 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1316/2659] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:10:49 2022] DELETE /identity/v3/users/aa56c83ab8494901b85d00d4f592e5ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:50.382191 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.400980 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f2613-934a-4d40-8149-8dc0d0613c01 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.490005 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.491720 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.492003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.492277 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] PATH_INFO: `/v3/projects/09c6a716650343c18e62bbe7235e48c8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.493484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c6a716650343c18e62bbe7235e48c8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.495705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.551719 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c6a716650343c18e62bbe7235e48c8', 'request_id': 'req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.561843 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c6a716650343c18e62bbe7235e48c8', 'request_id': 'req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.571273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c6a716650343c18e62bbe7235e48c8', 'request_id': 'req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.589235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] The token cache is being invalidate because project 09c6a716650343c18e62bbe7235e48c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:10:50.589739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.590557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-9e31b910-49b0-4b3e-a658-b36d8a6bf1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:10:50.592424 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1344/2660] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:10:50 2022] DELETE /identity/v3/projects/09c6a716650343c18e62bbe7235e48c8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:50.597738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.598029 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.598314 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.619506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:50.620834 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:50.698362 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1317/2661] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:10:50.705895 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.706190 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.706498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.725394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:50.726614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:50.734539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1345/2662] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:50.739040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.749415 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca17e39-ba49-4569-93fc-74a097228ebf None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.754264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.755115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.755384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.755847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.757217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:10:50.757608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.759402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.772374 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1318/2663] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:10:50.776798 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.785815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3f05e-856f-4c4c-964e-ad9f32d3d93d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.790418 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.791055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.791336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.791608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.792329 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.794222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.820183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1346/2664] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:10:50 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:10:50.824658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.835151 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500c133-973a-4d10-b77b-8e96a645e82f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.892882 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.893546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.893830 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.894118 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.894817 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.896712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.922642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1319/2665] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:10:50 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:10:50.926752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.937666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e3f98-b407-4620-9415-5c32454fa069 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.943756 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:50.954020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:50.954295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:50.954589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:50.955354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:50.956985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:50.969420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1347/2666] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:50.973435 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:50.988398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39099e-02d1-48f4-9a92-42bba4f0a9b9 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:50.999495 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:51.000684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:51.001134 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:51.001777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:51.002647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:51.004485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c1ae4-0119-498d-a082-e2f06da83b30 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:51.030861 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1320/2667] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:10:51.035778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:51.049399 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cf15a-ccf8-4536-a046-48f7dd0222fc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:51.057995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:51.060658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:51.060949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:51.061228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] PATH_INFO: `/v3/projects/057df79ceee746d7bbd932006cca3ce5/users/3b54ed811f2a46fc8c1a0235e06f9509/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:51.081680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057df79ceee746d7bbd932006cca3ce5, user_id=3b54ed811f2a46fc8c1a0235e06f9509, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:51.096823 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:51.123451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1348/2668] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:10:51 2022] PUT /identity/v3/projects/057df79ceee746d7bbd932006cca3ce5/users/3b54ed811f2a46fc8c1a0235e06f9509/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:10:51.124960 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09076a-2f19-4d05-9a6f-634422bbeeed None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:51.125235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09076a-2f19-4d05-9a6f-634422bbeeed None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:51.125513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09076a-2f19-4d05-9a6f-634422bbeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:51.149121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4a09076a-2f19-4d05-9a6f-634422bbeeed None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:51.150341 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4a09076a-2f19-4d05-9a6f-634422bbeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:51.233685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1321/2669] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:51.267261 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:51.275503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:51.351289 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2026e-4c1b-4403-80d1-1ccd2ac66879 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:51.355983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:51.356990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:51.357511 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:51.358002 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:51.369882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:51.371472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:51.384403 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1349/2670] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:55.878076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e21a-6360-4bf7-b13d-a9c735d9b847 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:55.878975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e21a-6360-4bf7-b13d-a9c735d9b847 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:55.879256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e21a-6360-4bf7-b13d-a9c735d9b847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:55.906312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-9461e21a-6360-4bf7-b13d-a9c735d9b847 None None] MFA Rules not processed for user `3b54ed811f2a46fc8c1a0235e06f9509`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:55.907561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9461e21a-6360-4bf7-b13d-a9c735d9b847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:55.993010 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1322/2671] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:10:56.000872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:10:56.002354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:56.085127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5754-7791-46a7-a19b-0dbc0ece9917 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:10:56.090637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:10:56.091291 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:56.091571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:56.091842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:56.100364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:10:56.101946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:10:56.111253 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1350/2672] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:10:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:10:57.344434 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a9ac-9efc-4b60-ba78-97605e917094 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:10:57.344958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a9ac-9efc-4b60-ba78-97605e917094 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:10:57.344958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a9ac-9efc-4b60-ba78-97605e917094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:10:57.362124 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3213a9ac-9efc-4b60-ba78-97605e917094 None None] MFA Rules not processed for user `3b54ed811f2a46fc8c1a0235e06f9509`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:10:57.363338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3213a9ac-9efc-4b60-ba78-97605e917094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:10:57.370917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1323/2673] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:10:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:11:30.165441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:11:30.166885 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:11:30.233965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58588c0-db08-4e3f-ba7d-88f02f87e377 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:11:30.238913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:11:30.239557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:11:30.240354 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:11:30.240627 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:11:30.242416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:11:30.254257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:11:30.256598 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75945f0c-f9db-4ad6-8149-ae96dab8874f service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:11:30.266990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1351/2674] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:11:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:11:38.692737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:11:38.693297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:11:38.693297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:11:38.709853 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] MFA Rules not processed for user `d4fb5382ce0d4e3694f4c434afc39d94`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:11:38.710981 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:11:38.762908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1324/2675] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:11:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:35.799880 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:35.800423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:35.800423 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:35.818488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] MFA Rules not processed for user `7fc10eb4012647b9874edca4482f7c14`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:35.819713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:35.883114 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1352/2676] 10.222.0.83 () {58 vars in 967 bytes} [Tue May 10 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:12:35.934535 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:35.935958 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:36.002546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9720b44-8893-4892-b2c7-5a8cae74a33c None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:36.007459 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a71e33d66744c208cd0bebb9d09d521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:36.008174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:36.008464 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:36.008744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:36.018209 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:36.019357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:36.029560 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1325/2677] 10.222.0.83 () {60 vars in 1384 bytes} [Tue May 10 15:12:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 15:12:40.102294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.104559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:40.119210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3f712-1295-48f6-9474-17df298853d9 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.124080 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.124944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.125223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.125492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] PATH_INFO: `/v3/users/d4fb5382ce0d4e3694f4c434afc39d94` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.126710 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fb5382ce0d4e3694f4c434afc39d94)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.128600 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.175539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fb5382ce0d4e3694f4c434afc39d94'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.176201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fb5382ce0d4e3694f4c434afc39d94'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.183050 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fb5382ce0d4e3694f4c434afc39d94'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.183765 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fb5382ce0d4e3694f4c434afc39d94'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.190775 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-33bafeed-48e1-4b34-b521-0fdfcc171b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fb5382ce0d4e3694f4c434afc39d94'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.207893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1353/2678] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:12:40 2022] DELETE /identity/v3/users/d4fb5382ce0d4e3694f4c434afc39d94 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:40.375795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.387888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4bdba-61a1-4864-8682-4d69c52aa802 service swift] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.444326 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.445213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.445493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.446380 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] PATH_INFO: `/v3/projects/870b2a5de87144f58bdd6aebe78a3e47` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.447574 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=870b2a5de87144f58bdd6aebe78a3e47)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.449545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.495966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '870b2a5de87144f58bdd6aebe78a3e47', 'request_id': 'req-7598fcac-1876-482d-8c67-579d3e5df7ec'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.501889 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '870b2a5de87144f58bdd6aebe78a3e47', 'request_id': 'req-7598fcac-1876-482d-8c67-579d3e5df7ec'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.508493 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '870b2a5de87144f58bdd6aebe78a3e47', 'request_id': 'req-7598fcac-1876-482d-8c67-579d3e5df7ec'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.523120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] The token cache is being invalidate because project 870b2a5de87144f58bdd6aebe78a3e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:12:40.523403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.524264 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7598fcac-1876-482d-8c67-579d3e5df7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:40.526095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1326/2679] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:12:40 2022] DELETE /identity/v3/projects/870b2a5de87144f58bdd6aebe78a3e47 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:40.532968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.533252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.533729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.552818 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:40.554205 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:40.634538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1354/2680] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:40.642538 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.642884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.643234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.662115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:40.663371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:40.672670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1327/2681] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:40.676926 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.687396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96199491-ec06-4933-be5e-47c88e4ffee8 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.692236 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.692913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.693196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.693470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.694229 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:12:40.694573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.696759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.714559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1355/2682] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:12:40.715797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.728975 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a5c68-2c50-42a3-9765-f369300988d4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.733838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.734570 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.734850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.735119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.735826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.737796 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.778572 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1328/2683] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:12:40 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:12:40.779701 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.791480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59f37-709a-4c8f-8cf4-981b1f7dd1cd admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.845713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.846326 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.846639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.846913 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.847632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.849492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.873705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1356/2684] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:12:40 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:12:40.874899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.884949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8c307-1cc0-4a56-a071-d03af9a3c684 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.889520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.890181 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.890484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.890761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.891497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.893096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.906051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1329/2685] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:40.907366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.918018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3c0c-95c8-41c1-8da7-849d4a4373f8 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.923960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.924651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.924938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.925244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.925965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.927592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf0055-da7f-45fe-820a-df0448f2ee92 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.938177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1357/2686] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:40.942372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:40.952180 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f6cb4-e9e9-41ac-a43a-d407bc13d6d0 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:40.956836 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:40.957738 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.958020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.958295 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] PATH_INFO: `/v3/projects/b89a1e184d314867bd22463ac32df4b5/users/2d6ea80876104ca0abdf693816c950fa/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:40.972421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89a1e184d314867bd22463ac32df4b5, user_id=2d6ea80876104ca0abdf693816c950fa, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:40.975110 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:40.988864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1330/2687] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:40 2022] PUT /identity/v3/projects/b89a1e184d314867bd22463ac32df4b5/users/2d6ea80876104ca0abdf693816c950fa/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:12:40.994480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:40.994779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:40.995061 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:41.012613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:41.013814 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:41.070802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1358/2688] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:41.080231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:41.082072 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:41.143925 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca4f-9ff4-447d-a583-bc5b77f276cb admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:41.147707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:41.148430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:41.149320 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:41.149677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:41.159865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:41.161398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:41.172357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1331/2689] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:12:42.863899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:42.866379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:42.881047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117a48d-2287-42c3-bf15-72ba7f613351 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:42.891776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:42.892532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:42.893139 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:42.893495 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] PATH_INFO: `/v3/users/4db1da4361c444da87400db06bbae3e4` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:42.894680 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db1da4361c444da87400db06bbae3e4)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:42.896641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:42.983151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db1da4361c444da87400db06bbae3e4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:42.987964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db1da4361c444da87400db06bbae3e4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:42.995960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db1da4361c444da87400db06bbae3e4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:42.997053 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db1da4361c444da87400db06bbae3e4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.006359 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db1da4361c444da87400db06bbae3e4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.027997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1359/2690] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:12:42 2022] DELETE /identity/v3/users/4db1da4361c444da87400db06bbae3e4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:12:43.223378 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:43.239739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6205112-7770-44b7-a0fe-afac01a0a852 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:43.308758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:43.309619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:43.309902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:43.310178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] PATH_INFO: `/v3/projects/69cc7901bd8e4ff68b7e943121e69580` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:43.311387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69cc7901bd8e4ff68b7e943121e69580)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:43.313276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:43.358965 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69cc7901bd8e4ff68b7e943121e69580', 'request_id': 'req-17831d3c-8fd8-40f5-ad84-932b992d8207'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.364226 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69cc7901bd8e4ff68b7e943121e69580', 'request_id': 'req-17831d3c-8fd8-40f5-ad84-932b992d8207'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.371143 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69cc7901bd8e4ff68b7e943121e69580', 'request_id': 'req-17831d3c-8fd8-40f5-ad84-932b992d8207'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.384068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] The token cache is being invalidate because project 69cc7901bd8e4ff68b7e943121e69580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:12:43.384357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.385000 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.386821 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1332/2691] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:12:43 2022] DELETE /identity/v3/projects/69cc7901bd8e4ff68b7e943121e69580 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:43.390563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:43.392189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:43.405279 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354c36-f7d0-40a3-9ab4-9d400a8fe12d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:43.470707 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:43.471657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:43.472478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:43.472750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] PATH_INFO: `/v3/users/36904454aa124cf1b3eaf9b15d65c8ab` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:43.474607 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=36904454aa124cf1b3eaf9b15d65c8ab)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:43.478084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:43.529990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36904454aa124cf1b3eaf9b15d65c8ab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.531163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36904454aa124cf1b3eaf9b15d65c8ab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.538911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36904454aa124cf1b3eaf9b15d65c8ab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.543349 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36904454aa124cf1b3eaf9b15d65c8ab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.555925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-81c6fafd-9ef2-4176-82ab-ddb1568ca9cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36904454aa124cf1b3eaf9b15d65c8ab'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.577948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1360/2692] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:12:43 2022] DELETE /identity/v3/users/36904454aa124cf1b3eaf9b15d65c8ab => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:43.780654 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:43.785208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:43.802251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831d3c-8fd8-40f5-ad84-932b992d8207 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:43.871379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:43.872266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:43.872536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:43.872816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] PATH_INFO: `/v3/projects/657130c9752e4748817a4bafcd53f33d` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:43.874476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] RBAC: Authorizing `identity:delete_project(project_id=657130c9752e4748817a4bafcd53f33d)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:43.876385 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:43.931304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657130c9752e4748817a4bafcd53f33d', 'request_id': 'req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.938771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657130c9752e4748817a4bafcd53f33d', 'request_id': 'req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.951900 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657130c9752e4748817a4bafcd53f33d', 'request_id': 'req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.971244 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] The token cache is being invalidate because project 657130c9752e4748817a4bafcd53f33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:12:43.971445 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.972514 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-8ba2fbf9-3aa1-490d-81c5-0031eb73289c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:12:43.975675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1333/2693] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:12:43 2022] DELETE /identity/v3/projects/657130c9752e4748817a4bafcd53f33d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:43.981019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:43.981538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:43.981538 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.007382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:44.009583 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:44.074651 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1361/2694] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:44.144292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.144686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.145082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.171160 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:44.172672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:44.185414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1334/2695] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:44.187073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.196733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380dff3-e1f4-4aaa-9069-8e031012bae2 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.202956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.202956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.203441 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.203807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.204632 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:12:44.205032 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.206989 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.226198 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1362/2696] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:12:44.227561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.245977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d92a9-d4b6-41b1-89ee-8a9750bd8149 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.255683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.256744 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.257358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.257873 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.258858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.264764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.333767 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1335/2697] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:12:44 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:12:44.338052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.352355 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f96893-41f0-40c6-94cb-78429aaef149 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.465633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.466982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.467157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.467668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.468539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.472360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.512968 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1363/2698] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:12:44 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:12:44.519584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.531622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbffcb-80b1-4c20-b146-2c5dd49eb9e5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.537649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.538579 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.539250 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.540635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.541899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.544004 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.566058 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1336/2699] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:44.573957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.587480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb02391-05d9-4365-b03e-fbaf1c4ccb8f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.593018 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.593744 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.594024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.594300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.595104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.596699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575c85b-f699-4c37-9d18-09137ca0426b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.610290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1364/2700] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:44.615241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.627135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73bd55-22a4-4b89-bcbc-d7d07621950c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.637644 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.638611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.638896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.639166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] PATH_INFO: `/v3/projects/68a7b38fce8444d9a3affb21189ee1b8/users/c7561439a3db4e3988ef5623c27c1aac/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.668499 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a7b38fce8444d9a3affb21189ee1b8, user_id=c7561439a3db4e3988ef5623c27c1aac, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.671390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.698480 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1337/2701] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:44 2022] PUT /identity/v3/projects/68a7b38fce8444d9a3affb21189ee1b8/users/c7561439a3db4e3988ef5623c27c1aac/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:44.699323 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dac6-6a49-45b8-b47c-e524f51a729a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.699624 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dac6-6a49-45b8-b47c-e524f51a729a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.699887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dac6-6a49-45b8-b47c-e524f51a729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.730867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-5561dac6-6a49-45b8-b47c-e524f51a729a None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:44.733841 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-5561dac6-6a49-45b8-b47c-e524f51a729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:44.823846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1365/2702] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:44.830632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:44.832856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:44.926857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611d786-39ea-4eb9-9edf-42f9cc81970f admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:44.932522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:44.933721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:44.934096 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:44.934510 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:44.949534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:44.951111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:44.964935 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1338/2703] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:12:45.853985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:45.854552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:45.854830 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:45.890764 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] MFA Rules not processed for user `2d6ea80876104ca0abdf693816c950fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:45.892247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:45.986042 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1366/2704] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:45.990783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:45.994816 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:46.009845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfffe-4c5a-49c9-a6a0-45aef2b224db service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.015815 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.016864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.017303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.017732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.018651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.020795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.059765 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1339/2705] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:12:45 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:12:46.063898 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:46.077637 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bbff8-eebb-4d95-bf9e-037aaa9f407f None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.154045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.155073 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.155531 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.155964 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.157388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.160642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.187562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1367/2706] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:12:46 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:12:46.191344 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:46.202793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb9aa-3cb6-40c3-99dd-94e6303133aa admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.207603 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.208275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.208546 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.208819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.209774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.212046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.232634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:46.233826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1340/2707] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:46.249167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ea5e8-f32d-46aa-b555-b1f6ba99da64 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.254852 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.256455 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.256899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.257341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] PATH_INFO: `/v3/projects/6975814d770046aca31973c6ba5fcf17/users/b9cfd35af7c8422785f9203313595c33/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.285379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6975814d770046aca31973c6ba5fcf17, user_id=b9cfd35af7c8422785f9203313595c33, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.288254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.311984 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1368/2708] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:46 2022] PUT /identity/v3/projects/6975814d770046aca31973c6ba5fcf17/users/b9cfd35af7c8422785f9203313595c33/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:12:46.312272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:46.324167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648542db-7558-453a-94d3-e24ffde60720 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.394715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.395406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.395691 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.396404 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.396703 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.398429 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a657656-12cb-4b42-ac81-800622ffe8dc admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.414263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1341/2709] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:46.417035 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:46.429150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763e73-54ad-4a85-84c8-dcaeda42dde1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:46.434362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:46.435380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:46.435732 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:46.436258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] PATH_INFO: `/v3/projects/6975814d770046aca31973c6ba5fcf17/users/b9cfd35af7c8422785f9203313595c33/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:46.438334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6975814d770046aca31973c6ba5fcf17, user_id=b9cfd35af7c8422785f9203313595c33, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:46.441242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:46.471313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1369/2710] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:46 2022] PUT /identity/v3/projects/6975814d770046aca31973c6ba5fcf17/users/b9cfd35af7c8422785f9203313595c33/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:50.549390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:50.549856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:50.549953 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:50.605903 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] MFA Rules not processed for user `c7561439a3db4e3988ef5623c27c1aac`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:50.607194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:50.723683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1342/2711] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:50.724197 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:50.852842 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540aeea-f4fc-4601-a997-7c2a7a3c4c9c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:50.870996 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:50.871791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:50.872317 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:50.872804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:50.876282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:50.878639 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:50.951517 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1370/2712] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:12:50 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:12:50.955304 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:50.971902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc2023-1cc1-4708-bc93-f6a781966eb0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:51.086850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:51.089458 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:51.090046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:51.090996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:51.093933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:51.097906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:51.154815 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:51.165426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1343/2713] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:12:50 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:12:51.171256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1730a-0ed6-4b99-a2c6-d73f4e84cf2d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:51.179222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:51.180992 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:51.182247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:51.182776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:51.183711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:51.186970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:51.204059 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1371/2714] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:51.208780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:51.224797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8198-4852-4c8b-b875-e7d5d6d6ee1e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:51.231446 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:51.232837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:51.232837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:51.233076 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] PATH_INFO: `/v3/projects/907a62abb86848f2b31584cd72208665/users/7a2fbced512b4d06ba4e19a3b267f93b/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:51.255962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=907a62abb86848f2b31584cd72208665, user_id=7a2fbced512b4d06ba4e19a3b267f93b, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:51.260693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:51.289068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1344/2715] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:51 2022] PUT /identity/v3/projects/907a62abb86848f2b31584cd72208665/users/7a2fbced512b4d06ba4e19a3b267f93b/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:12:51.293694 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:51.309364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253b797-0c5b-49e2-85c3-ec7bc2d5b9c4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:51.384470 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:51.385312 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:51.386421 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:51.386986 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:51.387873 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:51.389785 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1eb35-ec1e-4835-9521-375de18770a5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:51.405058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1372/2716] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:12:51.409389 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:51.420413 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e4f9-af1d-4295-b5ef-c7863b5cf4fd admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:51.426234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:51.427353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:51.427697 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:51.427881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] PATH_INFO: `/v3/projects/907a62abb86848f2b31584cd72208665/users/7a2fbced512b4d06ba4e19a3b267f93b/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:51.431865 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=907a62abb86848f2b31584cd72208665, user_id=7a2fbced512b4d06ba4e19a3b267f93b, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:51.434586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:51.445221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1345/2717] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:12:51 2022] PUT /identity/v3/projects/907a62abb86848f2b31584cd72208665/users/7a2fbced512b4d06ba4e19a3b267f93b/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:12:52.560886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e253c5-cdca-41ff-ac03-54b2f314592a None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:52.564001 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e253c5-cdca-41ff-ac03-54b2f314592a None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:52.564862 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e253c5-cdca-41ff-ac03-54b2f314592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:52.630193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-83e253c5-cdca-41ff-ac03-54b2f314592a None None] MFA Rules not processed for user `b9cfd35af7c8422785f9203313595c33`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:52.632574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-83e253c5-cdca-41ff-ac03-54b2f314592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:52.848757 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1373/2718] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:52.863645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:52.866039 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:53.028897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866502-6460-4db8-8d26-90b3a9420175 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:53.038665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:53.039711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:53.040166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:53.041898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:53.162916 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:53.165113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:53.184222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1346/2719] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:12:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:12:57.340264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2831-fefb-4557-b97f-9cea8271dd87 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:57.341122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2831-fefb-4557-b97f-9cea8271dd87 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:57.341590 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2831-fefb-4557-b97f-9cea8271dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:57.372957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-bcdf2831-fefb-4557-b97f-9cea8271dd87 None None] MFA Rules not processed for user `7a2fbced512b4d06ba4e19a3b267f93b`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:57.374734 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bcdf2831-fefb-4557-b97f-9cea8271dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:57.457216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1374/2720] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:12:57.480186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:12:57.481807 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:57.520915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cbf75-14da-484d-a195-b2b1f0b96b25 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:12:57.534819 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:12:57.536120 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:57.536484 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:57.536980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:57.626991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:12:57.628632 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5bd9f-e86c-4dd5-b0fe-0abb07eb94ae service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:12:57.640601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1347/2721] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:12:59.680695 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:12:59.681242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:12:59.681242 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:12:59.701020 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] MFA Rules not processed for user `c7561439a3db4e3988ef5623c27c1aac`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:12:59.702315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:12:59.715433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1375/2722] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:13:15.728939 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:15.729454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:15.729454 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:15.748016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] MFA Rules not processed for user `2d6ea80876104ca0abdf693816c950fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:13:15.749486 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:15.756265 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1348/2723] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:13:15.791925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:15.793299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:15.845711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5557e-e766-49be-8df9-fe55843cc54d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:15.849168 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:15.849795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:15.850077 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:15.850331 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:15.852561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:15.865230 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:15.866656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:15.878173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1376/2724] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:16.153647 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:16.205023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93f32d-b912-4108-92d6-1446c33d45dc None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:16.208507 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:16.209137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:16.209414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:16.209684 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:16.211895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:16.220940 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:16.222349 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:16.230541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1349/2725] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:16.661357 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:16.662786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:16.668503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:16.669901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:16.680218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115eb5-9d0b-46b5-b08e-c9987c798143 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:16.683857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:16.684491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:16.684780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:16.685056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:16.686789 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:16.700330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:16.701761 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:16.713166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1350/2726] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:13:16.731223 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d200-4508-4c62-a4b5-1ce5c99def54 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:16.736173 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:16.736811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:16.737092 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:16.737365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:16.739111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:16.750863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:16.752532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:16.762947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1377/2727] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:18.442271 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:18.452023 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856bdb4-045d-4311-8faa-785af7512f97 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:18.455869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:18.456646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:18.457114 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:18.457433 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:18.459383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:18.469964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:18.471628 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:18.481455 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1351/2728] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:22.577327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:22.594766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1912-dbb2-4d00-b7fd-23786d6ee514 service placement] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:22.601112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:22.602116 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:22.602767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:22.603288 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:22.605618 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:13:22.622098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:22.624265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:22.647930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1378/2729] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:32.225370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:32.233845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9541-5de1-41dc-92b8-7bc3bf715376 service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:32.237436 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:32.238066 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:32.238351 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:32.238662 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:32.247357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:32.249028 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1a4ed-2756-4bde-a668-205f9d92be24 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:32.258676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1352/2730] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:13:36.859305 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:13:36.868285 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d406f1-ecb0-42cb-89fa-e3f3cfb490e5 service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:13:36.871769 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:13:36.872391 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:13:36.872669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:13:36.872939 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:13:36.881699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:13:36.883611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:13:36.892597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1379/2731] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:14:23.202847 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:14:23.203417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:14:23.203417 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:14:23.220084 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] MFA Rules not processed for user `3b54ed811f2a46fc8c1a0235e06f9509`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:14:23.221363 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:14:23.273278 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1353/2732] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:14:32.270021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:14:32.283052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3139-aa8f-480b-81e9-ac5af4821609 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:14:32.288437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:14:32.289228 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:14:32.289393 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:14:32.289669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:14:32.291461 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:14:32.304420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:14:32.306023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:14:32.316768 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1380/2733] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:14:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:14:35.657719 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:14:35.670479 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56174530-ac2a-4b20-8105-d3ab849abcef None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:14:35.675927 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:14:35.676562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:14:35.676843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:14:35.677117 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:14:35.678902 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:14:35.690850 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:14:35.692414 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:14:35.703477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1354/2734] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:14:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:41.715584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:41.717585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:41.877782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64889d-7470-452b-8b01-97dac116d25f service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:41.882617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:41.883494 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:41.883784 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:41.884060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] PATH_INFO: `/v3/users/3b54ed811f2a46fc8c1a0235e06f9509` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:41.885267 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b54ed811f2a46fc8c1a0235e06f9509)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:41.887257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:41.939787 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b54ed811f2a46fc8c1a0235e06f9509'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:41.940449 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b54ed811f2a46fc8c1a0235e06f9509'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:41.947634 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b54ed811f2a46fc8c1a0235e06f9509'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:41.948271 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b54ed811f2a46fc8c1a0235e06f9509'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:41.956075 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-0b7fdd19-17d2-43a4-95f0-343148a16e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b54ed811f2a46fc8c1a0235e06f9509'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:41.973811 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1381/2735] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:15:41 2022] DELETE /identity/v3/users/3b54ed811f2a46fc8c1a0235e06f9509 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:42.141998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.180063 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731e869-4b76-4203-a9de-c9c629649113 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.234508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.235406 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.235692 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.235964 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] PATH_INFO: `/v3/projects/057df79ceee746d7bbd932006cca3ce5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.237139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057df79ceee746d7bbd932006cca3ce5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.239088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.284605 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057df79ceee746d7bbd932006cca3ce5', 'request_id': 'req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:42.289996 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057df79ceee746d7bbd932006cca3ce5', 'request_id': 'req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:42.296991 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057df79ceee746d7bbd932006cca3ce5', 'request_id': 'req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:42.311826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] The token cache is being invalidate because project 057df79ceee746d7bbd932006cca3ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:15:42.312113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:42.312731 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-791b15e7-10c2-45e7-8f33-1ccfa27c09f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:42.314266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1355/2736] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:15:42 2022] DELETE /identity/v3/projects/057df79ceee746d7bbd932006cca3ce5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:42.319204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.319491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.319767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.379404 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:42.380623 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:42.441445 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1382/2737] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:42.449989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.450272 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.450581 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.468559 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:42.469739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:42.477465 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1356/2738] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:42.482488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.491690 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf8d31-ef64-49a0-81cf-ecfbaf4325bc None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.496221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.496870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.497147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.497414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.498122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:15:42.498446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.500016 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.514571 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1383/2739] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:15:42.515712 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.524721 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dc712-5cee-4c81-b66f-289c94337b40 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.529859 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.530545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.530829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.531106 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.531791 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.533701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.560135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1357/2740] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:15:42.563489 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.573358 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeb2f3-33df-4768-80d2-2cf1422c71c4 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.629463 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.630146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.630448 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.630722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.631428 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.633406 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.655783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1384/2741] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:42.660082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.670810 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73174-7275-4c4a-b4ff-052a6d669766 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.676585 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.677270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.677551 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.677825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.678582 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.680390 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.691992 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1358/2742] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:42.695796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.706819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492920-3879-4e8b-8ff7-ab9d6deeff8e admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.711662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.712340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.712625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.712902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.713646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.715821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc64e12-3b47-4bb9-afe9-2b22405846d0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.727631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1385/2743] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:42.731952 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.742845 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838bdd-0e38-426c-b26d-498383963a0d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.747947 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.748971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.749322 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.749663 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] PATH_INFO: `/v3/projects/d18bf2a9676c4fa4ba1c320e9fb188cb/users/1eb562a963be40ffbc6f20b8ab3c32b6/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.768777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18bf2a9676c4fa4ba1c320e9fb188cb, user_id=1eb562a963be40ffbc6f20b8ab3c32b6, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:42.771757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:42.789195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1359/2744] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:42 2022] PUT /identity/v3/projects/d18bf2a9676c4fa4ba1c320e9fb188cb/users/1eb562a963be40ffbc6f20b8ab3c32b6/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:15:42.800482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.800883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.801224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.833907 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] MFA Rules not processed for user `1eb562a963be40ffbc6f20b8ab3c32b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:42.835340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:42.909662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1386/2745] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:42.916809 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:42.990695 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8f8a-4277-4841-b706-0fd2b6b4cb8e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:42.996553 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:42.997235 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:42.997519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:42.997795 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:42.998708 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.000913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.038673 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1360/2746] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:42 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:15:43.039767 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.052912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82fbb-ab9c-4841-89e3-66d4e3334755 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.112397 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.113074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.113356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.113662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.114348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.116282 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.137944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1387/2747] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:43 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:15:43.141881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.152677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a2f4-1b99-41a1-a640-0b279fbe6eee admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.157686 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.158355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.158676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.158955 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.159674 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.161277 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.172584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1361/2748] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:43.176832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.187594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fcde-960b-4372-af55-01ebeed09f72 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.192667 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.193611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.193894 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.194167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] PATH_INFO: `/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8/users/f1cc2c4dca5b4e14976261fee0568182/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.210347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8f6294dbdd4400ab2f115d9eb6ded8, user_id=f1cc2c4dca5b4e14976261fee0568182, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.213633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.228834 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1388/2749] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:43 2022] PUT /identity/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8/users/f1cc2c4dca5b4e14976261fee0568182/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:43.233577 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.244024 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daca97f-f35c-47b7-8bce-ee41194d0015 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.311369 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.312730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.313221 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.313912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.314885 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.316824 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c19d0-d004-4c70-b2a4-f850a50f565d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.330497 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1362/2750] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:43.334195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.344210 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa79e-6f2e-48e7-85d4-b6c7f5f53398 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.348801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.349665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.349940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.350208 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] PATH_INFO: `/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8/users/f1cc2c4dca5b4e14976261fee0568182/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.352163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8f6294dbdd4400ab2f115d9eb6ded8, user_id=f1cc2c4dca5b4e14976261fee0568182, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.354850 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.365957 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1389/2751] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:43 2022] PUT /identity/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8/users/f1cc2c4dca5b4e14976261fee0568182/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:43.377005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.377289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.377562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.403472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] MFA Rules not processed for user `f1cc2c4dca5b4e14976261fee0568182`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:43.404784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:43.470959 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1363/2752] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:43.482779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.484465 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:43.590298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b1cd-dd40-46cd-abd0-97d6e8fbe32f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.594527 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.595182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.595471 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.595742 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.606558 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.608037 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413ec87-f413-4130-a720-98bf86b9b7d8 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.623417 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1390/2753] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:43.731057 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:43.745070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e71ef-f4fc-4b9c-b3ce-2ad2cb93e298 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:43.749638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:43.750355 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.751009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.751358 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:43.822705 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:43.824405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:43.834020 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1364/2754] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:43.994960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:43.995468 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:43.995661 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:44.013997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] MFA Rules not processed for user `2d6ea80876104ca0abdf693816c950fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:44.015360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:44.072819 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1391/2755] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:44.382219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:44.383615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:44.457044 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc5ea5-d97c-43f1-a82a-1b72b4419cdf service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:44.460670 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:44.461308 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:44.461589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:44.461863 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:44.463617 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:44.473193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:44.474637 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334531-c6b7-43dd-8b10-e7fc013a1230 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:44.482312 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1365/2756] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:53.022251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:53.084547 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e377d-ca22-42a4-ae20-12585b873223 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:53.089218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:53.090678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:53.090965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:53.091234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] PATH_INFO: `/v3/users/1eb562a963be40ffbc6f20b8ab3c32b6` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:53.092429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb562a963be40ffbc6f20b8ab3c32b6)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:53.094334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:53.142104 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb562a963be40ffbc6f20b8ab3c32b6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.142740 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb562a963be40ffbc6f20b8ab3c32b6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.149407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb562a963be40ffbc6f20b8ab3c32b6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.150021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb562a963be40ffbc6f20b8ab3c32b6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.157807 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb562a963be40ffbc6f20b8ab3c32b6'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.175846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1392/2757] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:15:53 2022] DELETE /identity/v3/users/1eb562a963be40ffbc6f20b8ab3c32b6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:53.181051 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:53.181568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:53.181704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:53.199778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:53.200982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:53.258215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1366/2758] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:53.268336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:53.269836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:53.327240 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94432eca-829a-4939-ae56-4eb630519480 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:53.330725 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:53.331379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:53.331665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:53.331946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:53.341292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:53.342752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:53.355573 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1393/2759] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 15:15:53.612201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:53.613639 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:53.631387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdf649-1934-4b33-845a-341446e264d4 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:53.636713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:53.637584 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:53.637891 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:53.638242 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] PATH_INFO: `/v3/projects/d18bf2a9676c4fa4ba1c320e9fb188cb` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:53.639352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18bf2a9676c4fa4ba1c320e9fb188cb)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:53.641280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:53.686735 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18bf2a9676c4fa4ba1c320e9fb188cb', 'request_id': 'req-0769491e-e5eb-46f9-9235-f55820b2affb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.691989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18bf2a9676c4fa4ba1c320e9fb188cb', 'request_id': 'req-0769491e-e5eb-46f9-9235-f55820b2affb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.698048 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18bf2a9676c4fa4ba1c320e9fb188cb', 'request_id': 'req-0769491e-e5eb-46f9-9235-f55820b2affb'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.709734 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] The token cache is being invalidate because project d18bf2a9676c4fa4ba1c320e9fb188cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:15:53.710018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.710665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.712184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1367/2760] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:15:53 2022] DELETE /identity/v3/projects/d18bf2a9676c4fa4ba1c320e9fb188cb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:53.716633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:53.725914 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751c1f0-0185-4b56-a260-5e7d7ff0ea98 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:53.788041 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:53.789071 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:53.789407 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:53.789747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] PATH_INFO: `/v3/users/f1cc2c4dca5b4e14976261fee0568182` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:53.791257 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1cc2c4dca5b4e14976261fee0568182)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:53.794294 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:53.837946 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1cc2c4dca5b4e14976261fee0568182'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.838612 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1cc2c4dca5b4e14976261fee0568182'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.845731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1cc2c4dca5b4e14976261fee0568182'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.846365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1cc2c4dca5b4e14976261fee0568182'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.853367 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-7cd88ad3-f2db-427c-adcb-05744f7782d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1cc2c4dca5b4e14976261fee0568182'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:53.870640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1394/2761] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:15:53 2022] DELETE /identity/v3/users/f1cc2c4dca5b4e14976261fee0568182 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:54.259173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.261198 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:54.281237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769491e-e5eb-46f9-9235-f55820b2affb admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.355573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.356447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.356730 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.357005 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] PATH_INFO: `/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.358176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8f6294dbdd4400ab2f115d9eb6ded8)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.360121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.404906 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8f6294dbdd4400ab2f115d9eb6ded8', 'request_id': 'req-aa4cd504-7753-4375-8607-47f1c98139e0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:54.410635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8f6294dbdd4400ab2f115d9eb6ded8', 'request_id': 'req-aa4cd504-7753-4375-8607-47f1c98139e0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:54.417421 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8f6294dbdd4400ab2f115d9eb6ded8', 'request_id': 'req-aa4cd504-7753-4375-8607-47f1c98139e0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:54.432067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] The token cache is being invalidate because project cb8f6294dbdd4400ab2f115d9eb6ded8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:15:54.432631 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:54.433357 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aa4cd504-7753-4375-8607-47f1c98139e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:54.438284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1368/2762] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:15:54 2022] DELETE /identity/v3/projects/cb8f6294dbdd4400ab2f115d9eb6ded8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:54.439971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.440451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.440550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.459072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:54.460276 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:54.518772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1395/2763] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:54.526556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.527373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.527649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.545883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:54.546987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:54.555372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1369/2764] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:54.559711 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.569024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682c0d9-a366-4ffb-a790-3e222c2fe31b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.573484 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.574180 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.574488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.574858 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.575584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:15:54.575886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.577469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.589874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1396/2765] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:15:54.594067 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.603381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbed32-fb4a-4d6f-ae29-400f27d89a18 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.608949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.610050 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.610468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.610898 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.611604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.613508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.639128 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1370/2766] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:54.643379 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.653731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfcfd-45d7-4dc7-b667-f46c855f386d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.712846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.713510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.713786 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.714055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.715307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.717183 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.738084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1397/2767] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:54.741091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.751527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a543-2935-4b85-9023-6188708625d1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.756273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.756928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.757207 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.757477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.758216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.759888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.770485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1371/2768] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:15:54.775207 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.787325 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8881a5e-e686-47da-8a4a-40c487815ffc admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.792004 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.792658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.792936 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.793209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.793921 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.795550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316e87-e885-492d-8470-bba7e51e2c03 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.806273 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1398/2769] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:54.810463 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:54.822255 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c477cf5-d5f5-4d2c-bdab-fa0f48a2351e admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:54.827405 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:54.828281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.828562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.828832 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] PATH_INFO: `/v3/projects/b58cd53be4764c7287edd95368ee3456/users/dadeed0959d744b984fbf28270a04eba/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.843798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58cd53be4764c7287edd95368ee3456, user_id=dadeed0959d744b984fbf28270a04eba, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:54.847167 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:54.860549 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1372/2770] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:54 2022] PUT /identity/v3/projects/b58cd53be4764c7287edd95368ee3456/users/dadeed0959d744b984fbf28270a04eba/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:54.870886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:54.871234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:54.871539 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:54.898780 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] MFA Rules not processed for user `dadeed0959d744b984fbf28270a04eba`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:54.899969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:54.965557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1399/2771] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:54.971338 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.033747 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547595a-128f-4fde-bb11-dd50f45e91e3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.040280 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.040944 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.041227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.041503 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.042189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.044186 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.069545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1373/2772] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:54 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:55.073608 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.084584 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77689f5-ec43-44df-80bb-0e27c1e59bd6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.139252 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.139915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.140201 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.140476 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.141166 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.143138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.163927 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1400/2773] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:55 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:55.167970 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.178083 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c758017-7ffb-42dd-ba1b-e6d1e668367d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.182694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.183987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.184270 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.184541 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.185247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.187014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.198053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1374/2774] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:55.203017 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.212890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f7e53-0868-4fab-a454-8a72f5e38873 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.218013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.219012 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.219341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.219642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] PATH_INFO: `/v3/projects/e782523a26b943e1bd72e81bc2cde7d5/users/425175e071f04551a8854370c1736394/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.234502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e782523a26b943e1bd72e81bc2cde7d5, user_id=425175e071f04551a8854370c1736394, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.237292 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.250010 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1401/2775] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:55 2022] PUT /identity/v3/projects/e782523a26b943e1bd72e81bc2cde7d5/users/425175e071f04551a8854370c1736394/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:55.254262 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.264804 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca9d6-ae5b-4329-a740-f2fc7393cad3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.318432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.319095 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.319372 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.319704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.320370 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.321961 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160f22-c378-4096-825e-26d7049ce53a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.334896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1375/2776] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:55.336047 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.346628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ca06-82d7-4c62-b408-6ddc3a26e6b5 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.351337 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.352256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.352557 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.352835 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] PATH_INFO: `/v3/projects/e782523a26b943e1bd72e81bc2cde7d5/users/425175e071f04551a8854370c1736394/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.354849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e782523a26b943e1bd72e81bc2cde7d5, user_id=425175e071f04551a8854370c1736394, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.358089 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.368869 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1402/2777] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:55 2022] PUT /identity/v3/projects/e782523a26b943e1bd72e81bc2cde7d5/users/425175e071f04551a8854370c1736394/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:55.380635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.380957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.381232 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.410792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] MFA Rules not processed for user `425175e071f04551a8854370c1736394`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:55.412013 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:55.480980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1376/2778] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:55.492222 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.493999 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:55.550199 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3c4fd-de23-46e7-8415-82d37167014c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.553905 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.554596 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.554911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.555220 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.565185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.566655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1a7e5-a8f5-4013-aeee-87d8b4170255 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.577162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1403/2779] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:55.605737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.681980 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f4f54-8068-4ed3-bbe0-1161b622fc81 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.686620 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.687616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.687899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.688175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] PATH_INFO: `/v3/users/dadeed0959d744b984fbf28270a04eba` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.689432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dadeed0959d744b984fbf28270a04eba)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.691383 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.739512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dadeed0959d744b984fbf28270a04eba'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:55.746919 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dadeed0959d744b984fbf28270a04eba'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:55.747593 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dadeed0959d744b984fbf28270a04eba'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:55.748243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dadeed0959d744b984fbf28270a04eba'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:55.756794 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dadeed0959d744b984fbf28270a04eba'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:55.782249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1377/2780] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:15:55 2022] DELETE /identity/v3/users/dadeed0959d744b984fbf28270a04eba => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:55.794831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.795164 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.795479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.812552 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:55.813804 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:55.869987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1404/2781] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:15:55.880583 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:55.881979 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:55.937768 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121a958-49e2-46f8-b6e5-a33704268e85 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:55.941438 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:55.942082 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:55.942366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:55.942676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:55.952418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:55.953858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:55.963289 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1378/2782] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:56.219910 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:56.221362 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:56.232938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bc50c-effa-4fc1-9842-dd3a390f36ec None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:56.237510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:56.238373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:56.238688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:56.238965 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] PATH_INFO: `/v3/projects/b58cd53be4764c7287edd95368ee3456` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:56.240138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58cd53be4764c7287edd95368ee3456)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:56.242045 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:56.288629 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58cd53be4764c7287edd95368ee3456', 'request_id': 'req-701cbf90-5d9d-411a-82ec-511aee0fc0fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.294685 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58cd53be4764c7287edd95368ee3456', 'request_id': 'req-701cbf90-5d9d-411a-82ec-511aee0fc0fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.300203 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58cd53be4764c7287edd95368ee3456', 'request_id': 'req-701cbf90-5d9d-411a-82ec-511aee0fc0fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.312175 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] The token cache is being invalidate because project b58cd53be4764c7287edd95368ee3456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:15:56.312453 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.313063 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.314656 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1405/2783] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:15:56 2022] DELETE /identity/v3/projects/b58cd53be4764c7287edd95368ee3456 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:56.318672 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:56.328652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0b630-ad7f-4634-9be5-e40a11c56eaf service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:56.381523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:56.382411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:56.382701 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:56.382976 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] PATH_INFO: `/v3/users/425175e071f04551a8854370c1736394` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:56.384175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425175e071f04551a8854370c1736394)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:56.386103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:56.431469 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425175e071f04551a8854370c1736394'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.438904 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425175e071f04551a8854370c1736394'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.439573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425175e071f04551a8854370c1736394'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.440203 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425175e071f04551a8854370c1736394'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.447757 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-280a5455-2008-4048-b1d9-f3bb87ebefa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425175e071f04551a8854370c1736394'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.465046 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1379/2784] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:15:56 2022] DELETE /identity/v3/users/425175e071f04551a8854370c1736394 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:56.744501 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:56.746758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:56.769088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbf90-5d9d-411a-82ec-511aee0fc0fa admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:56.833327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:56.834302 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:56.834678 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:56.834883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] PATH_INFO: `/v3/projects/e782523a26b943e1bd72e81bc2cde7d5` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:56.836239 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e782523a26b943e1bd72e81bc2cde7d5)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:56.838130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:56.895027 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e782523a26b943e1bd72e81bc2cde7d5', 'request_id': 'req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.901163 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e782523a26b943e1bd72e81bc2cde7d5', 'request_id': 'req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.910350 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e782523a26b943e1bd72e81bc2cde7d5', 'request_id': 'req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.923182 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] The token cache is being invalidate because project e782523a26b943e1bd72e81bc2cde7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:15:56.923502 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.924121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-2ba3ce67-a577-4c03-beaa-a0a618a5b2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:15:56.925696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1406/2785] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:15:56 2022] DELETE /identity/v3/projects/e782523a26b943e1bd72e81bc2cde7d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:56.930485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:56.930771 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:56.931043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:56.950298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:56.951827 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:57.011416 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1380/2786] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:57.019385 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.019705 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.019983 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.038382 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:57.039585 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:57.047833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1407/2787] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:57.051841 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.061978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f7f21-d7b8-48a5-a418-6937254f7e49 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.066527 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.067176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.067457 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.067737 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.068452 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:15:57.068758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.070361 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.081995 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1381/2788] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:15:57.086153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.095859 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e86b0a-9127-4cf3-a214-bd3218049af6 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.100586 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.101269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.101559 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.101839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.102569 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.104562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.133051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1408/2789] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:57.134166 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.144408 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828a3d4-e6bb-4015-9624-47769f3295c3 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.201626 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.202212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.202652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.202928 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.203611 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.205496 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.224931 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1382/2790] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:57.229098 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.238990 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af5dc2-2c31-4a38-911a-c5940c3acff7 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.243562 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.244211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.244491 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.244766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.245478 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.247611 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.257945 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1409/2791] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:57.262139 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.271911 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531bc53-34f5-4dda-916f-13e6522005c1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.276860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.277512 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.277798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.278073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.278837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.280418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed8d97-e24a-45b0-93fb-20fb133252a1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.289966 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1383/2792] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:57.294081 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.303643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62dcf7-aaa5-43df-8d7e-ea81610a3770 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.308730 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.309622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.309949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.310247 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] PATH_INFO: `/v3/projects/8fdb0bb3f59a4d378902025c3fe621d2/users/45cf614364ca4a1eb18531460996b0af/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.324121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fdb0bb3f59a4d378902025c3fe621d2, user_id=45cf614364ca4a1eb18531460996b0af, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.326802 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.339870 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1410/2793] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:57 2022] PUT /identity/v3/projects/8fdb0bb3f59a4d378902025c3fe621d2/users/45cf614364ca4a1eb18531460996b0af/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:57.349895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.350381 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.350515 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.376473 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] MFA Rules not processed for user `45cf614364ca4a1eb18531460996b0af`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:57.377658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:57.438154 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1384/2794] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:57.442595 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.500731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ac8b-2bfa-497f-bd2a-88d8723ceb53 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.505528 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.506171 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.506477 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.506749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.507416 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.509287 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.533322 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1411/2795] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:57.537266 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.548260 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00080d-f3b8-4281-b968-69f7965e5057 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.601240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.601908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.602190 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.602490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.603201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.605727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.625339 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1385/2796] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/users => generated 302 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:15:57.629360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.640007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b95ae-9873-41b8-b008-e0de761cd574 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.644650 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.645319 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.645633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.645918 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.646729 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.648373 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.658649 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1412/2797] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:57.662491 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.672547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078abf4-fb64-4d1f-8627-081850b2c5bd admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.677353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.678256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.678566 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.678838 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] PATH_INFO: `/v3/projects/7bd1b23e078049c7b848fc353e743a4d/users/f0aad0ee260c496194ca539c9063fd61/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.694130 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd1b23e078049c7b848fc353e743a4d, user_id=f0aad0ee260c496194ca539c9063fd61, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.696869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.710683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1386/2798] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:57 2022] PUT /identity/v3/projects/7bd1b23e078049c7b848fc353e743a4d/users/f0aad0ee260c496194ca539c9063fd61/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:57.714762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.724831 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f385-ad9a-4b34-b085-3d3e3547531a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.781628 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.782290 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.782603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.782878 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.783594 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.785157 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a5dae-b277-442d-bf6e-af824c4458eb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.799749 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1413/2799] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:15:57.810633 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.822521 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225890f-bca7-4c0c-b73d-e0f8f4086044 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:57.827934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:57.829049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.829325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.829624 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] PATH_INFO: `/v3/projects/7bd1b23e078049c7b848fc353e743a4d/users/f0aad0ee260c496194ca539c9063fd61/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.831888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd1b23e078049c7b848fc353e743a4d, user_id=f0aad0ee260c496194ca539c9063fd61, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:57.834800 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:57.845728 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1387/2800] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:15:57 2022] PUT /identity/v3/projects/7bd1b23e078049c7b848fc353e743a4d/users/f0aad0ee260c496194ca539c9063fd61/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:15:57.857912 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:57.858206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:57.858532 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:57.884597 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] MFA Rules not processed for user `f0aad0ee260c496194ca539c9063fd61`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:57.885789 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:57.950072 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1414/2801] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:57.960614 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:57.962292 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:58.018987 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81886c-8ce7-4804-b2df-d747b61fb8d1 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:58.022687 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:58.023275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.023562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.023855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.033718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:58.035187 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc24f5-bcc9-4c73-b464-e5515840d848 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:58.045528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1388/2802] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:58.090563 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:58.091940 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:58.104146 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8b66-b64c-4d04-8ec3-68774e6979d3 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:58.107882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:58.108529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.108840 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.109151 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.167801 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:58.169280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ec41e-53eb-49a5-b9ff-708373dcb51a service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:58.178653 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1415/2803] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:58.284432 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.284625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.284905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] PATH_INFO: `/` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.286460 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1389/2804] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:15:58.293129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2af1-8d0f-4874-9e43-36a6b6ce203d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.293530 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2af1-8d0f-4874-9e43-36a6b6ce203d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.293931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2af1-8d0f-4874-9e43-36a6b6ce203d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.345336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-3e7e2af1-8d0f-4874-9e43-36a6b6ce203d None None] MFA Rules not processed for user `85f9abef52de40929e0bf2f8e28cd608`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:58.346688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-3e7e2af1-8d0f-4874-9e43-36a6b6ce203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:58.407068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1416/2805] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:15:58.420161 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:58.422241 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:58.435656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388448-f2ae-457e-950d-ef132697ab78 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:58.439724 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:58.440373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.440726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.441204 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.449215 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:58.450892 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34f95-289e-4811-a626-3fe51d86eb88 service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:58.459109 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1390/2806] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:15:58.549176 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad734b-ae4d-4691-990a-959f4fe3ddcb None None] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.549535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad734b-ae4d-4691-990a-959f4fe3ddcb None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.549833 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad734b-ae4d-4691-990a-959f4fe3ddcb None None] PATH_INFO: `/` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.551353 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1417/2807] 10.222.0.83 () {54 vars in 921 bytes} [Tue May 10 15:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 15:15:58.556883 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.557175 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.557447 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.575660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] MFA Rules not processed for user `5bb3df7bdd8847a1a3e17993b1f466a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:15:58.576843 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:15:58.646638 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1391/2808] 10.222.0.83 () {58 vars in 1018 bytes} [Tue May 10 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:15:58.657160 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad734b-ae4d-4691-990a-959f4fe3ddcb None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:15:58.668447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad734b-ae4d-4691-990a-959f4fe3ddcb None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:15:58.671944 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:15:58.672582 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:15:58.672893 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:15:58.673195 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:15:58.683766 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:15:58.685189 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bbac7-bee6-4791-b65b-22121b3c421c service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:15:58.695666 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1418/2809] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:01.510615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:01.569419 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccfae1-ae7a-4800-9378-a890df294a43 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:01.574508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:01.575373 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:01.575652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:01.575922 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] PATH_INFO: `/v3/users/45cf614364ca4a1eb18531460996b0af` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:01.577091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45cf614364ca4a1eb18531460996b0af)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:01.579074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:01.623135 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45cf614364ca4a1eb18531460996b0af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:01.629896 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45cf614364ca4a1eb18531460996b0af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:01.630592 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45cf614364ca4a1eb18531460996b0af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:01.631201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45cf614364ca4a1eb18531460996b0af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:01.638729 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45cf614364ca4a1eb18531460996b0af'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:01.655263 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1392/2810] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:01 2022] DELETE /identity/v3/users/45cf614364ca4a1eb18531460996b0af => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:01.659671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:01.660136 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:01.660235 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:01.677412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:01.678615 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:01.733377 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1419/2811] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:01.742914 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:01.744301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:01.803179 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b1b55-9188-43ae-817b-9323d291221d admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:01.807001 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:01.807645 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:01.807937 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:01.808700 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:01.817488 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:01.818957 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:01.828600 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1393/2812] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:02.180575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:02.182118 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:02.196472 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dc61b-b50e-4200-b539-e8ae9bd42c55 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:02.202997 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:02.204021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:02.204295 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:02.204576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] PATH_INFO: `/v3/projects/8fdb0bb3f59a4d378902025c3fe621d2` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:02.206310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fdb0bb3f59a4d378902025c3fe621d2)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:02.208251 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:02.259040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fdb0bb3f59a4d378902025c3fe621d2', 'request_id': 'req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.266021 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fdb0bb3f59a4d378902025c3fe621d2', 'request_id': 'req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.271696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fdb0bb3f59a4d378902025c3fe621d2', 'request_id': 'req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.283857 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] The token cache is being invalidate because project 8fdb0bb3f59a4d378902025c3fe621d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:02.284142 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.284747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.286269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1420/2813] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:02 2022] DELETE /identity/v3/projects/8fdb0bb3f59a4d378902025c3fe621d2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:02.290257 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:02.300474 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a23ea0-6bf8-4d1b-8a14-f98987969d8e service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:02.353694 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:02.354586 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:02.355398 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:02.355676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] PATH_INFO: `/v3/users/f0aad0ee260c496194ca539c9063fd61` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:02.356855 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0aad0ee260c496194ca539c9063fd61)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:02.358823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:02.404173 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0aad0ee260c496194ca539c9063fd61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.410664 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0aad0ee260c496194ca539c9063fd61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.411303 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0aad0ee260c496194ca539c9063fd61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.411895 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0aad0ee260c496194ca539c9063fd61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.419231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-3d0758f1-25b3-4706-8242-2854b0cfcb0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0aad0ee260c496194ca539c9063fd61'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.435236 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1394/2814] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:02 2022] DELETE /identity/v3/users/f0aad0ee260c496194ca539c9063fd61 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:02.773604 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:02.777250 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:02.793570 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40982b2e-dbf5-4286-8ed8-0b2cd6e63302 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:02.858200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:02.859105 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:02.859387 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:02.859658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] PATH_INFO: `/v3/projects/7bd1b23e078049c7b848fc353e743a4d` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:02.861068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd1b23e078049c7b848fc353e743a4d)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:02.863304 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:02.910550 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd1b23e078049c7b848fc353e743a4d', 'request_id': 'req-52592ed4-1067-45e1-9ff3-0e559a2d805d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.916882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd1b23e078049c7b848fc353e743a4d', 'request_id': 'req-52592ed4-1067-45e1-9ff3-0e559a2d805d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.922846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd1b23e078049c7b848fc353e743a4d', 'request_id': 'req-52592ed4-1067-45e1-9ff3-0e559a2d805d'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.936311 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] The token cache is being invalidate because project 7bd1b23e078049c7b848fc353e743a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:02.936737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.937366 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-52592ed4-1067-45e1-9ff3-0e559a2d805d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:02.938969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1421/2815] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:02 2022] DELETE /identity/v3/projects/7bd1b23e078049c7b848fc353e743a4d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:02.943837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:02.944330 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:02.944426 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:02.963716 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:02.964973 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:03.027033 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1395/2816] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:03.034721 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.035006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.035280 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.055663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:03.056941 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:03.066023 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1422/2817] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 15:16:03.070281 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.079982 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ad38-de6e-4590-bfff-6bd141d69e7f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.084920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.085542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.085826 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.086097 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.086867 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:16:03.087169 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.088772 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.103565 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1396/2818] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:16:03.104688 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.114132 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c39836-cb71-4bab-b69a-58f7d911d283 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.118772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.119414 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.119696 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.119969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.121218 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.123123 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.152296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1423/2819] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:16:03 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:16:03.156675 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.166784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4995b-ecaf-4a7a-9165-2281966cd980 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.225977 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.227043 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.227489 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.227698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.228353 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.231356 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.258658 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1397/2820] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:16:03 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:16:03.259836 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.270782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf1cb-b263-464e-95c7-007829a73967 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.279759 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.280931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.281219 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.281499 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.282234 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.283953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.295824 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1424/2821] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:16:03.300132 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.312648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297944-8356-4689-aca3-5bd01d46ee4c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.317359 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.318212 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.318542 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.318814 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.319557 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.321240 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957d788-4000-4e27-bc27-286472b65b75 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.334423 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1398/2822] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:03.336308 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.347987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a63388-9a36-4cb1-b4ab-555068428a72 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.352960 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.354429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.354429 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.354644 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] PATH_INFO: `/v3/projects/2efd618c8e774c71abea46e97809cbab/users/8794ab9e28cf4a6c99a2b9c1fc93e835/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.368792 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efd618c8e774c71abea46e97809cbab, user_id=8794ab9e28cf4a6c99a2b9c1fc93e835, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.372074 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.385613 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1425/2823] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:03 2022] PUT /identity/v3/projects/2efd618c8e774c71abea46e97809cbab/users/8794ab9e28cf4a6c99a2b9c1fc93e835/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:03.396102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c718f-161f-47d1-840e-138285959e02 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.396387 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c718f-161f-47d1-840e-138285959e02 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.396661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c718f-161f-47d1-840e-138285959e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.421920 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-131c718f-161f-47d1-840e-138285959e02 None None] MFA Rules not processed for user `8794ab9e28cf4a6c99a2b9c1fc93e835`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:03.423137 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-131c718f-161f-47d1-840e-138285959e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:03.483547 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1399/2824] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:03.495872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:03.497575 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:03.552953 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf18d7-78b9-45ac-b81f-ec9605c9a791 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:03.556603 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:03.557244 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:03.557522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:03.557794 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:03.566981 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:03.568432 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17938bd6-78dc-4cc4-ade9-51ec0d14e1e1 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:03.577395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1426/2825] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:22.060485 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c718f-161f-47d1-840e-138285959e02 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.129769 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c718f-161f-47d1-840e-138285959e02 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:22.134269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:22.135159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.135440 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.135741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] PATH_INFO: `/v3/users/8794ab9e28cf4a6c99a2b9c1fc93e835` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.136915 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8794ab9e28cf4a6c99a2b9c1fc93e835)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:22.138858 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:22.186243 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8794ab9e28cf4a6c99a2b9c1fc93e835'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.192698 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8794ab9e28cf4a6c99a2b9c1fc93e835'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.193348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8794ab9e28cf4a6c99a2b9c1fc93e835'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.193949 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8794ab9e28cf4a6c99a2b9c1fc93e835'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.201133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8794ab9e28cf4a6c99a2b9c1fc93e835'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.218607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1400/2826] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:22 2022] DELETE /identity/v3/users/8794ab9e28cf4a6c99a2b9c1fc93e835 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:22.223109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.223619 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.223760 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.240185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:22.241341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:22.297733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1427/2827] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:22.307200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.309088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:22.364334 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3ccc1-657c-4443-a962-7149b43e9445 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:22.367888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:22.368522 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.368802 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.369074 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.378715 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:22.380140 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc381d0-ca38-490f-8489-1c641326a07a service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:22.389812 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1401/2828] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:22.642216 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.643763 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:22.655645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73c7a4-6964-4ffb-9bd1-ce979bc82c1d None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:22.660129 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:22.660985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.661269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.661549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] PATH_INFO: `/v3/projects/2efd618c8e774c71abea46e97809cbab` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.662762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efd618c8e774c71abea46e97809cbab)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:22.664655 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:22.712109 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efd618c8e774c71abea46e97809cbab', 'request_id': 'req-e5d89a91-3fbd-417f-8678-2e0704f32019'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.718206 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efd618c8e774c71abea46e97809cbab', 'request_id': 'req-e5d89a91-3fbd-417f-8678-2e0704f32019'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.723492 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efd618c8e774c71abea46e97809cbab', 'request_id': 'req-e5d89a91-3fbd-417f-8678-2e0704f32019'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.736360 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] The token cache is being invalidate because project 2efd618c8e774c71abea46e97809cbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:22.736645 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.737356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-e5d89a91-3fbd-417f-8678-2e0704f32019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:22.738969 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1428/2829] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:22 2022] DELETE /identity/v3/projects/2efd618c8e774c71abea46e97809cbab => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:22.743962 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.744259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.744543 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.762881 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:22.764062 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:22.821727 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1402/2830] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:22.829860 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.830150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.830456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.848396 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:22.849616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:22.858095 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1429/2831] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:22.862234 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.871780 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586b9a0-f349-4d62-a85f-084c47746964 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:22.876599 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:22.877237 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.877516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.877783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.878523 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79937) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:16:22.878829 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:22.880442 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:22.891251 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1403/2832] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:16:22.895522 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.904845 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb9d8f-e961-4101-9504-147669599d40 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:22.909420 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:22.910068 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:22.910348 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:22.910657 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:22.911351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:22.913458 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:22.939117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1430/2833] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:16:22 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:16:22.943403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:22.953604 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03132b09-a7d7-45c6-8c20-29b3357450a5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.008539 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.009133 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.009439 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.009714 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.010411 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.012352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.032068 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1404/2834] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:16:22 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:16:23.036140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.046437 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d683a-effb-40e7-87b0-c7807c8739f1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.051048 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.051700 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.051982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.052256 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.052970 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.054578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.065122 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1431/2835] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:23.069092 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.079622 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d671f-648e-4bec-adc1-8c6daf824068 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.084273 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.084918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.085202 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.085477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.086191 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.087813 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67454d-cd17-4d14-a701-1341cda8df8c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.097293 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1405/2836] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:23.101161 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.111392 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b007f-8930-4ee8-b396-afa722bb10f6 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.115974 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.116839 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.117121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.117395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] PATH_INFO: `/v3/projects/009d4c4209a143aa8155efa0930c2277/users/06a2da39c2af4cf695b4de7702635077/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.131817 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009d4c4209a143aa8155efa0930c2277, user_id=06a2da39c2af4cf695b4de7702635077, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.134488 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.146443 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1432/2837] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:23 2022] PUT /identity/v3/projects/009d4c4209a143aa8155efa0930c2277/users/06a2da39c2af4cf695b4de7702635077/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:23.156377 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.156682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.156963 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.183225 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] MFA Rules not processed for user `06a2da39c2af4cf695b4de7702635077`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:23.184562 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:23.252365 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1406/2838] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:23.257007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.315112 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f49e1-0ec6-4823-9802-90c2cff42ed1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.319646 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.320293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.320577 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.320855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.321554 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.323985 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.348140 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1433/2839] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:16:23.351894 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.362216 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0730f4-9ecb-4c8f-ad6d-cf7729766684 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.414269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.414951 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.415233 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.415506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.416183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.418616 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.437401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1407/2840] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/users => generated 288 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:16:23.441254 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.451915 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73b00-b335-451e-9bba-45bc0f65995a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.456412 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.457058 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.457341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.457617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.458327 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.459947 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.470299 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1434/2841] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:23.473785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.483897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fda2b-c85b-4e9b-a0e7-ce86c07cc52c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.488476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.489119 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.489401 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.489676 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.490396 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.491994 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23cdd-8c80-42ea-bed8-230e218f0dd5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.501711 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1408/2842] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:23.505606 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.515352 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b58eb-4d31-4a09-90f8-db14df22dead admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.519888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.520758 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] REQUEST_METHOD: `PUT` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.521040 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.521315 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] PATH_INFO: `/v3/projects/baadd8177d0049dc8d0c0e6d4a336455/users/402283a2412f4cc6ac2e5d6c0d33b05b/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.535663 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baadd8177d0049dc8d0c0e6d4a336455, user_id=402283a2412f4cc6ac2e5d6c0d33b05b, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.538293 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.550296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1435/2843] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:23 2022] PUT /identity/v3/projects/baadd8177d0049dc8d0c0e6d4a336455/users/402283a2412f4cc6ac2e5d6c0d33b05b/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:23.611806 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.612079 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.612367 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.639472 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] MFA Rules not processed for user `402283a2412f4cc6ac2e5d6c0d33b05b`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:23.640671 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:23.701501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1409/2844] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:23.705871 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.764526 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4b64f-0ec1-4cc6-94d1-c73515216370 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.769185 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.769827 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.770110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.770405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] PATH_INFO: `/v3/projects` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.771127 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.773036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.798535 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1436/2845] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:16:23.802537 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.813009 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38d053-0c4e-44ac-a67e-e681ebde7118 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.864403 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.865053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.865328 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.865598 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] PATH_INFO: `/v3/users` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.866275 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.868189 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.888073 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1410/2846] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:16:23 2022] POST /identity/v3/users => generated 294 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 15:16:23.892019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.902204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c427-e8ed-40dc-9a59-173e755b039a admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.906776 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.907433 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.907713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.907988 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.908706 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.910278 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.921540 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1437/2847] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:23.925860 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.936623 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a3b01-1673-411a-b8f3-e016e0e7f109 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:23.941183 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:23.942052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:23.942331 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:23.942683 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] PATH_INFO: `/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b/users/01c1af59943e47ac9775cf1dafb956bf/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:23.956619 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924539c1f9b54bc5b33a4bc0c8fa634b, user_id=01c1af59943e47ac9775cf1dafb956bf, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:23.959400 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:23.972297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1411/2848] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:23 2022] PUT /identity/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b/users/01c1af59943e47ac9775cf1dafb956bf/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:23.976364 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:23.986576 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf87d5-561b-4fe6-b59c-43895e8610dd admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:24.040561 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:24.041224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:24.041510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:24.041783 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:24.043283 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:24.044883 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeed274-7036-40eb-8c67-659cd076810b admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:24.054972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1438/2849] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 15:16:24.060371 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:24.070948 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b03c76-824c-4b2d-a421-7e1318d051b2 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:24.075677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:24.076556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:24.076839 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:24.077115 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] PATH_INFO: `/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b/users/01c1af59943e47ac9775cf1dafb956bf/roles/57e23e25b8c046ba882d07965d852386` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:24.079103 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=924539c1f9b54bc5b33a4bc0c8fa634b, user_id=01c1af59943e47ac9775cf1dafb956bf, role_id=57e23e25b8c046ba882d07965d852386)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:24.081797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:24.093666 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1412/2850] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:24 2022] PUT /identity/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b/users/01c1af59943e47ac9775cf1dafb956bf/roles/57e23e25b8c046ba882d07965d852386 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:24.111610 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:24.111911 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:24.112162 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:24.139088 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] MFA Rules not processed for user `01c1af59943e47ac9775cf1dafb956bf`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:24.140200 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:24.205750 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1439/2851] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:24.217012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:24.218835 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:24.273758 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5208b-8b69-43e4-bb98-80d34bf1c96c admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:24.278049 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:24.278717 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:24.278998 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:24.279269 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:24.332019 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:24.333471 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:24.342053 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1413/2852] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:25.805900 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:25.807343 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:25.820145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2509f98-dd05-4ad4-811c-f9ff230b97a2 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:25.823633 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:25.824264 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:25.824545 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:25.824813 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:25.878727 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:25.880130 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:25.889060 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1440/2853] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:26.385552 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:26.394561 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eda8-149d-4cac-8bee-1eb6cb2a9d77 service cinder] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:26.398174 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:26.398777 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:26.399056 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:26.399329 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:26.408534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:26.409945 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23516-19ea-4914-9a10-39b367d4a3b2 service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:26.419297 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1414/2854] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:30.224622 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:30.283855 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655a50-363f-4a67-959e-4ab828b78ec8 service cinder] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:30.288973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:30.289846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:30.290124 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:30.290405 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] PATH_INFO: `/v3/users/06a2da39c2af4cf695b4de7702635077` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:30.291578 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a2da39c2af4cf695b4de7702635077)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:30.295479 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:30.339560 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a2da39c2af4cf695b4de7702635077'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.340196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a2da39c2af4cf695b4de7702635077'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.346713 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a2da39c2af4cf695b4de7702635077'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.347556 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a2da39c2af4cf695b4de7702635077'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.354731 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a2da39c2af4cf695b4de7702635077'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.371752 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1441/2855] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:30 2022] DELETE /identity/v3/users/06a2da39c2af4cf695b4de7702635077 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:16:30.376164 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:30.376899 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:30.377012 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:30.395556 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:30.396732 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:30.451660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1415/2856] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:30.461388 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:30.462829 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:30.517473 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400de628-9fa8-4791-b8a0-ef7a2e7495df admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:30.521102 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:30.521739 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:30.522024 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:30.522300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:30.531712 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:30.533117 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:30.542351 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1442/2857] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:30.798856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:30.800311 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:30.812309 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2beb4-f85a-4782-ab3e-87e8dd14bbab None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:30.817038 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:30.817901 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:30.818176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:30.818476 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] PATH_INFO: `/v3/projects/009d4c4209a143aa8155efa0930c2277` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:30.819648 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=009d4c4209a143aa8155efa0930c2277)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:30.821651 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:30.866888 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009d4c4209a143aa8155efa0930c2277', 'request_id': 'req-54d20dce-afe2-40c7-808b-fcfef4d88bfe'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.872782 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009d4c4209a143aa8155efa0930c2277', 'request_id': 'req-54d20dce-afe2-40c7-808b-fcfef4d88bfe'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.878942 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009d4c4209a143aa8155efa0930c2277', 'request_id': 'req-54d20dce-afe2-40c7-808b-fcfef4d88bfe'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.890501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] The token cache is being invalidate because project 009d4c4209a143aa8155efa0930c2277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:30.890783 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.891393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:30.892905 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1416/2858] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:30 2022] DELETE /identity/v3/projects/009d4c4209a143aa8155efa0930c2277 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:30.896899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:30.906592 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a969-7fd9-4fc8-a8db-fa83c983e763 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:30.961791 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:30.962692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:30.962972 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:30.963245 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] PATH_INFO: `/v3/users/402283a2412f4cc6ac2e5d6c0d33b05b` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:30.964896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402283a2412f4cc6ac2e5d6c0d33b05b)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:30.966832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:31.009762 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402283a2412f4cc6ac2e5d6c0d33b05b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.010342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402283a2412f4cc6ac2e5d6c0d33b05b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.017658 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402283a2412f4cc6ac2e5d6c0d33b05b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.018296 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402283a2412f4cc6ac2e5d6c0d33b05b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.024949 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402283a2412f4cc6ac2e5d6c0d33b05b'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.041681 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1443/2859] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:30 2022] DELETE /identity/v3/users/402283a2412f4cc6ac2e5d6c0d33b05b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:31.304249 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:31.306069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:31.323665 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20dce-afe2-40c7-808b-fcfef4d88bfe admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:31.391642 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:31.392494 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:31.392774 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:31.393047 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] PATH_INFO: `/v3/projects/baadd8177d0049dc8d0c0e6d4a336455` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:31.394229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=baadd8177d0049dc8d0c0e6d4a336455)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:31.396159 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:31.438787 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baadd8177d0049dc8d0c0e6d4a336455', 'request_id': 'req-337dbab0-044d-4d0f-9868-0b9ba7371e0a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.444217 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baadd8177d0049dc8d0c0e6d4a336455', 'request_id': 'req-337dbab0-044d-4d0f-9868-0b9ba7371e0a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.450575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baadd8177d0049dc8d0c0e6d4a336455', 'request_id': 'req-337dbab0-044d-4d0f-9868-0b9ba7371e0a'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.461972 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] The token cache is being invalidate because project baadd8177d0049dc8d0c0e6d4a336455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:31.462274 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.462913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.464513 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1417/2860] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:31 2022] DELETE /identity/v3/projects/baadd8177d0049dc8d0c0e6d4a336455 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:31.469019 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:31.470483 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:31.483262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76770662-4b0f-4668-9c92-1cee07879b22 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:31.538145 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:31.539051 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:31.539341 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:31.539617 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] PATH_INFO: `/v3/users/01c1af59943e47ac9775cf1dafb956bf` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:31.540886 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c1af59943e47ac9775cf1dafb956bf)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:31.542952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:31.586549 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c1af59943e47ac9775cf1dafb956bf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.587193 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c1af59943e47ac9775cf1dafb956bf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.594202 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c1af59943e47ac9775cf1dafb956bf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.594851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c1af59943e47ac9775cf1dafb956bf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.601503 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-5c36a57e-84e1-4b5b-876d-e7734d61c5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c1af59943e47ac9775cf1dafb956bf'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:31.617795 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1444/2861] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:31 2022] DELETE /identity/v3/users/01c1af59943e47ac9775cf1dafb956bf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:31.880091 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:31.881615 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:31.895753 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbab0-044d-4d0f-9868-0b9ba7371e0a admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:31.951707 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:31.952575 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:31.952856 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:31.953131 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] PATH_INFO: `/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:31.954299 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=924539c1f9b54bc5b33a4bc0c8fa634b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:31.956256 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:31.999718 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924539c1f9b54bc5b33a4bc0c8fa634b', 'request_id': 'req-869934dd-25f1-4d59-aa47-7812d589d13c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:32.005195 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924539c1f9b54bc5b33a4bc0c8fa634b', 'request_id': 'req-869934dd-25f1-4d59-aa47-7812d589d13c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:32.011568 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924539c1f9b54bc5b33a4bc0c8fa634b', 'request_id': 'req-869934dd-25f1-4d59-aa47-7812d589d13c'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:32.023227 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] The token cache is being invalidate because project 924539c1f9b54bc5b33a4bc0c8fa634b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:32.023509 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:32.024116 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-869934dd-25f1-4d59-aa47-7812d589d13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:32.025646 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1418/2862] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:31 2022] DELETE /identity/v3/projects/924539c1f9b54bc5b33a4bc0c8fa634b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:32.031542 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.031925 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.032211 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.050046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:32.051260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:32.107342 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1445/2863] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:32.117393 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.117912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.118477 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.139054 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:32.140177 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:32.148064 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1419/2864] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:32.152328 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.161938 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e72c7-f60e-45df-87d4-6bdb1f93e76b None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.166456 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.167747 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.168026 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.168298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] PATH_INFO: `/v3/domains` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.169006 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79936) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 15:16:32.169307 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.170903 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.182011 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1446/2865] 10.222.0.83 () {58 vars in 1152 bytes} [Tue May 10 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 15:16:32.186755 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.195693 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc355d-f8ca-47ec-8702-546f1992a4ff None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.200989 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.201634 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.201909 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.202184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] PATH_INFO: `/v3/projects` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.202917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.204818 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.237825 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1420/2866] 10.222.0.83 () {60 vars in 1152 bytes} [Tue May 10 15:16:32 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:16:32.244867 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.254971 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3f47-34fe-46ff-8cf6-4684d590520d admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.308226 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.308881 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.309167 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.309446 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] PATH_INFO: `/v3/users` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.310120 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.312108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.332029 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1447/2867] 10.222.0.83 () {60 vars in 1143 bytes} [Tue May 10 15:16:32 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 15:16:32.336247 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.346785 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39576a54-e0f1-4232-9b23-307ee6050678 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.351379 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.352025 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.352305 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.352578 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.353284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.354912 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.364936 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1421/2868] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:32.368669 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.378614 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc171be-6f29-415c-ae57-ab709c85568f admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.383186 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.383832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.384115 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.384394 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.385110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.387224 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288f77c-b0de-41b3-b89d-ce0cd07e7cfb admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.398684 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1448/2869] 10.222.0.83 () {58 vars in 1121 bytes} [Tue May 10 15:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 15:16:32.402974 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.412680 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29819-b4f8-45c5-a9f6-b8651f349d4b admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.417739 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.418635 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.418917 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.419193 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] PATH_INFO: `/v3/projects/145da26dd473456fb82c956ab8240093/users/73c21dfa0b184e6094a8fe05ee4b398f/roles/edf562e307eb48899cd024cddf6d7be3` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.433213 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145da26dd473456fb82c956ab8240093, user_id=73c21dfa0b184e6094a8fe05ee4b398f, role_id=edf562e307eb48899cd024cddf6d7be3)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.436517 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.449348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1422/2870] 10.222.0.83 () {60 vars in 1482 bytes} [Tue May 10 15:16:32 2022] PUT /identity/v3/projects/145da26dd473456fb82c956ab8240093/users/73c21dfa0b184e6094a8fe05ee4b398f/roles/edf562e307eb48899cd024cddf6d7be3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:32.460709 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] REQUEST_METHOD: `POST` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.461003 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.461286 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.487447 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.auth.core [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] MFA Rules not processed for user `73c21dfa0b184e6094a8fe05ee4b398f`. Rule list: `[]` (Enabled: `True`). {{(pid=79936) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:32.488642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:32.550269 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1449/2871] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:32.560879 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:32.562589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:32.616797 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159efd2-34b0-4490-b470-caa2a45e65d5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:32.620704 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f00c2803cc6a4e4a982d05cd419e5301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:32.621348 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:32.621636 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:32.621908 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:32.630776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:32.632201 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8921f5-c7bc-46bd-9f70-c0f73cd0e7ef service cinder] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:32.640869 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1423/2872] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:42.475031 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:42.533380 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea26546-43fa-49dc-962c-7e2871d08d47 None None] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:42.537796 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:42.538668 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:42.538948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:42.539221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] PATH_INFO: `/v3/users/73c21dfa0b184e6094a8fe05ee4b398f` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:42.540519 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c21dfa0b184e6094a8fe05ee4b398f)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:42.543055 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:42.587631 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c21dfa0b184e6094a8fe05ee4b398f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:42.588265 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c21dfa0b184e6094a8fe05ee4b398f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:42.594979 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c21dfa0b184e6094a8fe05ee4b398f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:42.595625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c21dfa0b184e6094a8fe05ee4b398f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:42.602480 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c21dfa0b184e6094a8fe05ee4b398f'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:42.618931 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1450/2873] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:16:42 2022] DELETE /identity/v3/users/73c21dfa0b184e6094a8fe05ee4b398f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:16:42.624652 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:42.625178 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:42.625301 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:42.642764 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] MFA Rules not processed for user `ce068d36274d4c2ab89128932ddb806d`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:16:42.643954 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:42.700608 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1424/2874] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:16:42.710356 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:42.711748 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:42.766778 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5c8a-2f2a-484e-9d26-262ad6c398d0 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:42.770300 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:42.771084 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:42.771368 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:42.771643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:42.780589 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:42.782013 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:42.791111 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1451/2875] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:16:43.050776 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:16:43.052184 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:16:43.064078 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e40ca-3808-4baf-ba53-ba68f2e6f9e0 None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:16:43.069060 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:16:43.069934 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:16:43.070210 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:16:43.070516 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] PATH_INFO: `/v3/projects/145da26dd473456fb82c956ab8240093` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:16:43.071745 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145da26dd473456fb82c956ab8240093)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:16:43.073685 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:16:43.120142 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145da26dd473456fb82c956ab8240093', 'request_id': 'req-cc3475b6-e1c9-44e0-a818-f9782fc905b0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:43.125259 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145da26dd473456fb82c956ab8240093', 'request_id': 'req-cc3475b6-e1c9-44e0-a818-f9782fc905b0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:43.131668 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145da26dd473456fb82c956ab8240093', 'request_id': 'req-cc3475b6-e1c9-44e0-a818-f9782fc905b0'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:43.154573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] The token cache is being invalidate because project 145da26dd473456fb82c956ab8240093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:16:43.154864 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:43.155482 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-cc3475b6-e1c9-44e0-a818-f9782fc905b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:16:43.157018 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1425/2876] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:16:43 2022] DELETE /identity/v3/projects/145da26dd473456fb82c956ab8240093 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:17:06.471716 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:06.474616 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:06.564692 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf046-4e02-4dcb-9a3c-43c84f49101d service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:06.621177 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:06.621877 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:06.622262 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:06.622671 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:06.624868 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:06.636170 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:06.637733 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:06.647451 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1452/2877] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:17:42.159069 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] REQUEST_METHOD: `POST` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:42.159656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:42.159656 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:42.179284 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.auth.core [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] MFA Rules not processed for user `2d6ea80876104ca0abdf693816c950fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79937) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 15:17:42.180726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:42.244601 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1426/2878] 10.222.0.83 () {58 vars in 957 bytes} [Tue May 10 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 15:17:45.231310 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:45.233674 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:45.296523 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb012a7e-a0fc-4c9e-b2a1-8916de21a16b service placement] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:45.300672 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:45.301643 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:45.302125 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:45.302640 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:45.304737 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:45.395334 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:45.397196 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:45.409336 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1453/2879] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:17:46.367418 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:46.369071 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:46.383121 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa615c8a-7bd3-41dd-93e6-21522575559f None None] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:46.388682 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:46.389555 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:46.389837 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:46.390111 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] PATH_INFO: `/v3/users/c7561439a3db4e3988ef5623c27c1aac` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:46.391384 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7561439a3db4e3988ef5623c27c1aac)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:46.393534 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:46.437325 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7561439a3db4e3988ef5623c27c1aac'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.444102 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7561439a3db4e3988ef5623c27c1aac'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.444778 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7561439a3db4e3988ef5623c27c1aac'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.445366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7561439a3db4e3988ef5623c27c1aac'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.453545 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7561439a3db4e3988ef5623c27c1aac'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.470366 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1427/2880] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:17:46 2022] DELETE /identity/v3/users/c7561439a3db4e3988ef5623c27c1aac => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:17:46.719564 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:46.734030 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18320db-a05a-4c8a-81e7-85567ece858b service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:46.803194 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:46.804110 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:46.804390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:46.804662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] PATH_INFO: `/v3/projects/68a7b38fce8444d9a3affb21189ee1b8` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:46.805851 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a7b38fce8444d9a3affb21189ee1b8)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:46.807832 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:46.853635 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a7b38fce8444d9a3affb21189ee1b8', 'request_id': 'req-c0b71add-da8b-44a8-8a7c-4c21db56b832'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.860121 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a7b38fce8444d9a3affb21189ee1b8', 'request_id': 'req-c0b71add-da8b-44a8-8a7c-4c21db56b832'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.865411 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a7b38fce8444d9a3affb21189ee1b8', 'request_id': 'req-c0b71add-da8b-44a8-8a7c-4c21db56b832'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.878849 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] The token cache is being invalidate because project 68a7b38fce8444d9a3affb21189ee1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:17:46.879138 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.879782 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:46.881574 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1454/2881] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:17:46 2022] DELETE /identity/v3/projects/68a7b38fce8444d9a3affb21189ee1b8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:17:46.886176 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:46.887713 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:46.901222 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10b79e-b225-44a2-a7f7-c70936ebdca9 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:46.967030 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:46.967999 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:46.968276 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:46.968550 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] PATH_INFO: `/v3/users/7a2fbced512b4d06ba4e19a3b267f93b` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:46.969741 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2fbced512b4d06ba4e19a3b267f93b)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:46.971726 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:47.017857 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2fbced512b4d06ba4e19a3b267f93b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.024500 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2fbced512b4d06ba4e19a3b267f93b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.025196 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2fbced512b4d06ba4e19a3b267f93b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.025793 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2fbced512b4d06ba4e19a3b267f93b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.033040 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2fbced512b4d06ba4e19a3b267f93b'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.049884 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1428/2882] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:17:46 2022] DELETE /identity/v3/users/7a2fbced512b4d06ba4e19a3b267f93b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:17:47.220340 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:47.221779 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:47.237204 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b71add-da8b-44a8-8a7c-4c21db56b832 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:47.293534 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:47.294415 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:47.294704 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:47.294982 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] PATH_INFO: `/v3/projects/907a62abb86848f2b31584cd72208665` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:47.296153 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=907a62abb86848f2b31584cd72208665)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:47.298179 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:47.342260 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '907a62abb86848f2b31584cd72208665', 'request_id': 'req-554f7091-aa50-44da-abaf-08078e3b3eb2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.348156 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '907a62abb86848f2b31584cd72208665', 'request_id': 'req-554f7091-aa50-44da-abaf-08078e3b3eb2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.353298 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '907a62abb86848f2b31584cd72208665', 'request_id': 'req-554f7091-aa50-44da-abaf-08078e3b3eb2'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.364959 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] The token cache is being invalidate because project 907a62abb86848f2b31584cd72208665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79936) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:17:47.365238 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.365846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:17:47.367398 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1455/2883] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:17:47 2022] DELETE /identity/v3/projects/907a62abb86848f2b31584cd72208665 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:17:53.485823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:17:53.487298 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:53.577840 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057f8a-f603-4608-90ed-ffeffacf85c5 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:17:53.638395 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:17:53.639070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:17:53.639352 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:17:53.639627 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:17:53.641391 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:17:53.699016 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:17:53.700677 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:17:53.709660 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1429/2884] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:08.707591 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:08.709052 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:08.762529 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f7091-aa50-44da-abaf-08078e3b3eb2 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:08.766007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:08.766665 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:08.766948 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:08.767221 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:08.768934 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:08.777896 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:08.779318 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:08.786973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1456/2885] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:33.152229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:33.162897 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6562b-68a9-4d8d-b78f-7f23c0031cfb service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:33.167580 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:33.168219 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:33.168528 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:33.168801 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:33.170625 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:33.184498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:33.185918 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:33.197088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1430/2886] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:34.369482 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:34.371036 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:34.381543 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87e2c-91b0-449d-b8c3-b6402bb46546 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:34.385846 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:34.386533 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:34.386821 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:34.387150 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:34.389108 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:34.404879 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:34.406313 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:34.417662 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1457/2887] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:35.223017 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:35.224589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:35.238763 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1b8a-cc7d-49bc-bec1-4f35bb638ebb service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:35.244589 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:35.245501 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:35.245784 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:35.246052 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:35.247808 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:35.260913 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:35.262518 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:35.273506 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1431/2888] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:39.735902 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:39.748872 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9468b-d56d-418a-bb21-abb3c35b7b56 service neutron] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:39.755548 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb3df7bdd8847a1a3e17993b1f466a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:39.756213 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:39.756498 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:39.756772 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:39.759887 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:39.772952 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:39.774890 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:39.786248 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1458/2889] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:45.166525 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:45.175760 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8797f-adeb-4aeb-b5ac-8e46af2a5735 service nova] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:45.179279 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85f9abef52de40929e0bf2f8e28cd608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:45.179933 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] REQUEST_METHOD: `GET` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:45.180218 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:45.180490 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:45.182229 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:45.244093 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:45.245536 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:45.255200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1432/2890] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:19:51.523525 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:51.524973 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:51.540159 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ea71-d55b-4143-aba2-fa39dec474ae service nova] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:51.544722 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:51.545588 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:51.545874 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:51.546147 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] PATH_INFO: `/v3/users/2d6ea80876104ca0abdf693816c950fa` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:51.547442 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6ea80876104ca0abdf693816c950fa)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:51.549395 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:51.597258 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6ea80876104ca0abdf693816c950fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.597899 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6ea80876104ca0abdf693816c950fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.605930 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6ea80876104ca0abdf693816c950fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.606642 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6ea80876104ca0abdf693816c950fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.613609 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6ea80876104ca0abdf693816c950fa'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.630963 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1459/2891] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:19:51 2022] DELETE /identity/v3/users/2d6ea80876104ca0abdf693816c950fa => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:19:51.807228 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:51.819823 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219ac61-6d56-4d0c-8c50-49f79585a7ce service neutron] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:51.875430 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:51.876231 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:51.876519 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:51.876798 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] PATH_INFO: `/v3/projects/b89a1e184d314867bd22463ac32df4b5` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:51.877978 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89a1e184d314867bd22463ac32df4b5)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:51.880208 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:51.925498 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89a1e184d314867bd22463ac32df4b5', 'request_id': 'req-7629208c-8886-470b-b1dd-e3021d025414'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.931946 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89a1e184d314867bd22463ac32df4b5', 'request_id': 'req-7629208c-8886-470b-b1dd-e3021d025414'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.939088 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89a1e184d314867bd22463ac32df4b5', 'request_id': 'req-7629208c-8886-470b-b1dd-e3021d025414'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.951573 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] The token cache is being invalidate because project b89a1e184d314867bd22463ac32df4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:19:51.951877 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.952520 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:51.954070 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1433/2892] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:19:51 2022] DELETE /identity/v3/projects/b89a1e184d314867bd22463ac32df4b5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:19:51.957995 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:51.959469 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:51.972601 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba7732-c0ba-4a1c-99d8-7e40e2631f3c admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:52.027347 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:52.028209 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:52.028497 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:52.028774 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] PATH_INFO: `/v3/users/b9cfd35af7c8422785f9203313595c33` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:52.029956 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9cfd35af7c8422785f9203313595c33)` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:52.031888 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:52.076365 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9cfd35af7c8422785f9203313595c33'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.077046 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9cfd35af7c8422785f9203313595c33'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.083863 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9cfd35af7c8422785f9203313595c33'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.084510 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9cfd35af7c8422785f9203313595c33'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.092882 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.notifications [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9cfd35af7c8422785f9203313595c33'} {{(pid=79936) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.112509 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1460/2893] 10.222.0.83 () {58 vars in 1223 bytes} [Tue May 10 15:19:51 2022] DELETE /identity/v3/users/b9cfd35af7c8422785f9203313595c33 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:19:52.275122 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Authenticating user token {{(pid=79937) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:19:52.276607 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.fernet_utils [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79937) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:19:52.289649 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629208c-8886-470b-b1dd-e3021d025414 admin admin] Validating token access rules against request {{(pid=79937) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:19:52.346014 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce068d36274d4c2ab89128932ddb806d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06d48372060e456ea0c20a517b83a4d3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79937) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:19:52.346910 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:19:52.347194 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:19:52.347468 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] PATH_INFO: `/v3/projects/6975814d770046aca31973c6ba5fcf17` {{(pid=79937) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:19:52.348661 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6975814d770046aca31973c6ba5fcf17)` {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:19:52.351113 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] RBAC: Authorization granted {{(pid=79937) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:19:52.396200 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6975814d770046aca31973c6ba5fcf17', 'request_id': 'req-098f342f-e9b7-48e2-845d-03ba8539e657'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.402099 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6975814d770046aca31973c6ba5fcf17', 'request_id': 'req-098f342f-e9b7-48e2-845d-03ba8539e657'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.408165 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6975814d770046aca31973c6ba5fcf17', 'request_id': 'req-098f342f-e9b7-48e2-845d-03ba8539e657'} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.420508 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] The token cache is being invalidate because project 6975814d770046aca31973c6ba5fcf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79937) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:19:52.420792 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.421402 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: DEBUG keystone.notifications [None req-098f342f-e9b7-48e2-845d-03ba8539e657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79937) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:19:52.422971 dv-cinder-839628-2-iscsi devstack@keystone.service[79937]: [pid: 79937|app: 0|req: 1434/2894] 10.222.0.83 () {58 vars in 1232 bytes} [Tue May 10 15:19:52 2022] DELETE /identity/v3/projects/6975814d770046aca31973c6ba5fcf17 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:22:11.419987 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Authenticating user token {{(pid=79936) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:22:11.421390 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:22:11.488154 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93079018-f966-4d59-afc0-f1a2a3d90ae1 admin admin] Validating token access rules against request {{(pid=79936) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:22:11.544699 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c876294f0a400f8bcd833377df454c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d331a421ec347a8ad32180fbf3f223b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79936) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:22:11.545345 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] REQUEST_METHOD: `GET` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:22:11.545625 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] SCRIPT_NAME: `/identity` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:22:11.545897 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79936) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:22:11.547641 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.fernet_utils [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79936) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:22:11.558993 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:22:11.560546 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fc38f-680a-44b1-a210-2b3f4a5a1a40 service placement] RBAC: Authorization granted {{(pid=79936) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:22:11.570007 dv-cinder-839628-2-iscsi devstack@keystone.service[79936]: [pid: 79936|app: 0|req: 1461/2895] 10.222.0.83 () {60 vars in 1365 bytes} [Tue May 10 15:22:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)